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Study  of  the  performance  of  a  teleradiology  system  in  five  hospitals  and 
dirties  in  the  Washington,  D.C.  area.  The  purpose  of  the  study  was  to  assess 
the  utility  of  teleradiology  in  medical  practice  in  small  health  care 
facilities  that  do  not  have  a  full-time  staff  radiologist. 
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SUMMARY  OF  FINDINGS 


A,  INTRODUCTION 

\V  Teleradiology  is  an  automated  system  whereby  an  electronic 
representation  of  an  X-ray  image  is  transmitted  via  telephone  wires 
from  one  location  to  another,  j  The  TRIMIS  Program  Office  (Department 
of  Defense)  and  the  Bureau  oijF  Radiological  Health  (Public  Health 
Service)  have  conducted  a  six-month  field  trial  of  a  Teleradiology 
system.  The  system,  which  was  designed  by  the  MITRE  Corporation,  was 
studied  to  evaluate  its  performance  in  a  routine  medical  practice 
setting. 

In  the  fall  of  1980,  four  small  clinics  in  the  Washington,  D.C. 
area  were  chosen  as  transmitter  sites  for  the  trial  and  a  large 
radiology  department  was  selected  as  the  central  receiver  site. 
Training  for  site  personnel  began  in  the  fall  of  1981.  The  system  was 
installed  in  January  of  1982,  becoming  fully  operational  in  late 
March,  and  remaining  so  through  June.  Baseline  (pre-implementation) 
data  collection  for  the  functional  evaluation  of  the  Teleradiology 
system  was  conducted  by  the  Bureau  of  Radiological  Health  for  two 
weeks  in  February  of  1981  and  by  Arthur  D.  Little,  Inc.  for  two  weeks 
in  September  of  that  year.  Post-implementation  data  were  collected  by 
Arthur  D.  Little,  Inc.  for  two  weeks  in  June  of  1982. 

The  Teleradiology  system  was  experimental  and,  hence,  was  not 
used  as  a  replacement  for  normal  film  interpretation  of  X-ray  exams 
during  the  field  trial.  The  "manual"  systems  used  by  each  of  the 
transmitter  clinics  continued  to  be  used  in  parallel  with 
Teleradiology.  Because  of  the  parallel  use  of  Teleradiology  and 
"manual"  methods  for  obtaining  interpretations,  many  of  the  data 
collected  for  the  functional  evaluation  were  designed  to  estimate  the 
system's  potential  impact,  rather  than  to  directly  measure  its  impact 
in  the  experimental  setting. 

Data  were  collected  concerning: 

e  the  potential  impact  of  the  system  on  patient  care; 

e  the  acceptability  of  the  system  to  users; 
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•  th«  potential  feasibility  of  using  the  system  in  routine 
practice  settings;  and 

e  the  potential  inpact  of  the  system  on  costs  of  transaltter 
site  X-ray  services. 

B.  THE  STUDY  SETTING 

The  central  receiver  site  for  the  Teleradiology  field  trial  ms 
the  Radiology  Department  at  Malcolm  Grow  Medical  Center  at  Andrews  Air 
Force  Base  in  Maryland.  During  the  field  trial,  this  large  radiology 
department  continued,  for  the  most  part,  to  operate  as  it  had  before. 
The  Teleradiology  equipment  was  physically  separated  from  the  other 
viewing  rooms  in  the  department;  the  transcriptionlsts  who  used  the 
system  were  especially  hired  for  the  project;  and  the  radiologists  who 
interpreted  teleradiology  Images  were  recruited  from  several  hospitals 
in  the  area  and  used  the  system  on  a  part-time,  scheduled,  volunteer 
basis. 

The  four  medical  facilities  used  as  transmitter  sites  during  the 
field  trial  were: 

e  Bolling  Air  Force  Base  Clinic,  Washington,  D.C. ; 

e  Fort  Detrick  Army  Clinic,  Frederick,  Maryland; 

e  Patuxent  Naval  Air  Station  Hospital,  Lexington  Park, 

Maryland;  and 

e  Central  Virginia  Community  Health  Center,  New  Canton, 

Virginia. 

These  clinics  and  their  X-ray  departments  are  quite  small  and  are 
oriented  primarily  toward  the  delivery  of  outpatient  services.  The 
clinics  vary  in  workload  from  21,000  outpatient  visits  per  year  to 
85,000.  Their  X-ray  departments  range  from  having  one  technician,  one 
X-ray  room  and  examining  1,400  patients  per  year  to  four  technicians, 
two  rooms  and  7,000  patients  per  year.  Each  of  the  four  sites  has 
standard  arrangements  with  speci-ic  medical  centers  for  secondary  and 
tertiary  care  referrals.  Their  distance  to  these  medical  centers 
varies  from  10  to  90  miles.  Normally,  two  of  the  four  clinics  send 
theirs  X-ray  films  by  courier  to  medical  centers  for  Interpretation; 
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om  employs  a  part-time  radiologist  to  perform  film  interpretations ; 
sad  ttaa  fourth  aonda  films  av ay  every  other  weak,  employing  a  visiting 
radiologist  during  the  off-weeks. 

During  the  field  trial •  the  transmitter  departments  continued  to 
have  their  films  read  "manually,"  as  they  had  before.  At  each  of  four 
of  the  transmitter  sites*  the  regular  technicians  were  trained  to  use 
the  Teleradiology  system  and  did  operate  it.  However*  at  three  of  the 
sites*  additional  temporary  personnel  were  also  hired  to  accommodate 
system  use. 


C.  RESULTS:  REAL  AND  POTENTIAL  IMPACTS 

1.  Impact  on  Patient  Care 

The  potential  Impact  of  Teleradiology  on  patient  care  was  studied 
in  two  ways.  First*  turnaround  time  of  interpretation  reports  was 
measured  for  all  exams  ordered  during  the  two  study  periods  (before 
and  after  the  Teleradiology  system  was  installed).  And  second,  a 
study  was  conducted  of  how  radiographs  and  X-ray  Interpretations  were 
used  in  patient  care  at  the  transmitter  facilities.  All  data 
regarding  turnaround  tine  and  patient  care  impacts  were  collected 
using  a  self-administered  survey  form,  which  followed  each  X-ray 
through  the  various  stages  of  the  X-ray  request /report  cycle  and  was 
completed  by  clinic  staff.  Data  were  collected  regarding  a  total  of 
418  patients  (453  exams)  before  system  Implementation  and  for  618 
patients  (695  exams)  during  system  operation. 
a.  Turnaround  Time;  X-Ray  Request /Report  Cycle 

Most  of  the  X-rays  sc  each  of  the  four  clinics  and  during  both 
data  collection  periods  were  performed  within  two  hours  of  being 
ordered.  However,  delays  of  several  days  often  occurred  between  the 
exam's  performance  and  its  interpretation  by  a  radiologist  and  between 
this  interpretation  and  its  review  by  the  referring  provider. 

The  total  mean  turnaround  time  required  for  "manual"  film 
Interpretation  reports  before  system  implementation  was  found  to  range 
from  88  hours  to  108  hours  at  the  four  study  sites.  During  system 
operation,  mean  turnaround  time  for  "manual"  film  interpretation 
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reports  mi  aama what  longer,  ranging,  at  tha  the  four  aitaa,  from  121 
to  233  hoar  a.  Thia  incraaae  in  turnaround  tine  may  ba  aaaoeiatad 
with: 

a  tha  parallel  uaa  of  tha  two  interpretation  sodalities 
during  the  post-implementation  data  collection  period;  and 

a  miscellaneous  breakdowns  in  the  "manual"  interpretation 
systems  during  the  post-implementation  data  collection 
period . 

The  total  mean  turnaround  tine  required  for  automated 
Teleradiology  interpretations  was  even  longer  than  that  for  manual 
services.  This  mean  varied  from  181  to  212  hours  at  the  four  sites. 
Delays  for  receipt  of  Teleradiology  interpretations  were  largely 
attributable  to  the  fact  that  the  system  was  only  used  experimentally : 

a  the  system  did  not  Initially  function  reliably; 

a  X-ray  images  were  not  always  input  regularly  or  on  a  daily 
basis; 

a  radiologists  were  generally  available  to  perform  Teleradiology 
interpretations  only  during  morning  hours  five  days  each  week; 

a  "manual"  interpretation  reports  were  often  received  prior 
to  Teleradiology  interpretations  of  the  same  exam,  and, 
hence,  were  not  always  read  promptly. 

In  order  to  project  the  impact  of  a  refined  Teleradiologv  system 
in  routine  use  (a  non-experlmental  setting),  it  is  assumed  that  the 
system  would  be  implemented  and  utilized  quite  differently.  For 
example: 

a  a  non-experlmental  system  would  probably  be  more  reliable; 

a  if  the  system  were  used  routinely,  a  protocol  would 
probably  be  established  for  Inputting  films  into  the 
system  regularly; 

a  a  radiologist  would  probably  be  available  on  a  full-time 
basis;  and 

a  no  parallel  "manual"  system  would  be  available. 

Based  on  these  assumptions,  the  data  collected  during  the  post- 
implementation  period  suggest  that  in  a  non-experlmental  situation. 
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turnaround  tine  for  teleradiology  interpretation  reporta  could  be  Bach 
shorter  than  that  observed  during  the  field  trial.  Inputting  of  films 
requires  approximately  10  minutes  per  exam,  and  telephone  lime 
transmission  takes  an  additional  15  to  30  minutes  per  exam.  As  soon 
as  transmission  is  complete,  interpretation  may  be  performed,  and 
either  the  telephone  or  the  tele-typewriter  may  be  used  to 
communicate  findings  Immediately  to  remote  providers.  He  project  that 
a  24-hour  total  turnaround  would  be  possible  for  routine  exams  and 
that  a  1-hour  turnaround  could  be  accomplished  for  STAT  exams. 

The  field  trial  figures  demonstrate,  however,  that  installation 
of  the  teleradiology  technology  does  not.  Itself,  result  in  reduced 
turnaround  time.  Although  system  operation  is  neither  extremely 
time-consuming  nor  very  complex,  for  the  system  to  be  used 
effectively,  protocols  must  be  established  for  inputting  films 
regularly,  radiologists  must  be  available  to  interpret  images  on  a 
routine  basis,  and,  as  under  any  reporting  system,  if  one’s  goal  is  to 
minimize  time  delays,  reports  must  be  delivered  to  providers  promptly 
and  read  by  providers  upon  receipt. 
b.  Impact  on  Patient  Care 

Data  were  collected  concerning  the  types  of  X-rays  performed 
during  the  two  study  periods  at  the  transmitter  site  X-ray 
departments.  Also,  for  each  X-ray  ordered  during  the  study  periods, 
referring  providers  were  asked  to  answer  three  different  questions 
concerning  the  use  of  interpretations  in  patient  care.  These 
questions  concerned  the  relative  significance  of  prompt  receipt  of  a 
radiologist's  interpretation;  the  role  of  X-ray  film  viewing  in 
patient  care  decisions;  and  the  role  of  the  radiologist's  report  in 
these  decisions.  Little  variation  existed  between  the  two  data 
collection  periods  in  the  types  of  exams  performed  and  the  patterns  of 
provider  response  to  "patient  care"  questions.  Hence,  these  data  have 
been  pooled  for  presentation. 

The  X-ray  exams  performed  at  the  study  sites  were  grouped  into 
four  categories  corresponding  to  various  clinical  uses: 
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•  Routine  physical  cheat  exams; 

•  Eaetgency  exaas  associated  with  acute  trauaa; 

s  Diagnostic  exaas  not  associated  with  acute  trauma;  and 

s  X-rays  taken  "for  the  record"  or  as  follow-up  exaas. 

Tbs  case  aix  in  the  X-ray  departments  at  each  of  the  transmitter 
facilities  is  quite  limited,  as  would  be  expected  in  primary  care 
clinics.  Also,  the  distribution  of  X-rays  performed  at  the  military 
sites  is  somewhat  different  from  that  at  the  civilian  clinic.  Central 
Virginia  Community  Health  Center  (CVCHC).  During  the  two  study 
periods,  23X  of  the  X-rays  performed  at  the  military  sites  were 
associated  with  routine  physicals  (compared  with  6Z  at  CVCHC);  40% 
were  associated  with  acute  trauma  (compared  with  28%  at  CVCHC) ; 

17X  were  other  "diagnostic"  exams  (compared  with  50%  at  CVCHC);  and 
20%  were  performed  "for  the  record"  or  as  follow-up  procedures  (versus 
16%  at  CVCHC). 

At  the  time  that  the  exam  was  requested,  providers  were  asked  to 
categorize  how  significant  a  prompt  receipt  of  a  radiologist's 
interpretation  would  be  in  patient  care.  In  73%  of  cases,  providers 
indicated  that  prompt  receipt  of  a  radiologist's  interpretation  would 
have  some  effect  on  their  opinions  or  decisions  regarding  patient 
cars.  Timely  interpretation  receipt  was  considered  "very  significant: 
essential  to  patient  care  decisions"  in  only  8%  of  cases.  Rapid 
interpretation  turnaround  was  felt  to  be  most  important  for  exams 
associated  with  trauma  and  other  diagnostic  exams  and  least 
significant  for  routine  physicals. 

After  X-rays  were  performed  at  the  transmitter  facilities,  the 
requesting  providers  viewed  the  films  themselves  in  a  majority  of 
cases.  Providers  were  most  likely  to  review  exams  associated  with 
acute  trauma  (77%),  followed  by  non-emergency  exams  that  they 
considered  diagnostic  (69%)  and  radiographs  performed  "for  the  record" 
or  as  follow-up  procedures  (62%).  They  seldom  viewed  routine  physical 
chest  exams  (10%). 
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If  and  when  the  referring  provider  viewed  the  films  that  he  had 
ordered,  he  was  asked  to  categorize  how  he  felt  this  viewing  had 
affected  his  handling  of  the  case.  Usually  (in  65%  of  cases)  the 
viewing  of  radiographs  served  to  "increase  the  clinical  confidence"  of 
providers.  Sometimes  —  primarily  in  trauma  cases  or  for  other 
diagnostic  exams  —  providers  reported  that  the  film  viewing  had  had  a 
major  effect  on  handling  of  the  case  (16%  and  21%  of  cases, 
respectively).  It  was  least  likely  to  have  had  a  major  effect  for 
routine  physicals  (4%  of  cases). 

At  the  time  the  provider  reviewed  the  radiologist's  film  or 
Teleradiology  interpretation  report,  he  was  asked  what  effect  the 
specialist's  report  had  made  on  patient  treatment /disposition 
decisions.  At  the  transmitter  sites  radiologists'  Interpretations 
were  almost  always  received  several  days  after  the  X-ray  had  been 
performed  and  the  patient  treated  and  sent  home  (see  above).  Reports 
received  after  so  long  a  delay  would  not  be  expected  to  have  much 
effect  on  patient  care  unless  their  findings  differed  substantially 
from  those  made  earlier  by  the  referring  provider.  Indeed,  by  the 
time  radiologists'  reports  were  reviewed,  they  were  felt  to  have  had 
no  effect  on  patient  care  for  42%  of  cases.  Forty-three  percent  of 
radiologists'  reports  were  felt  to  have  "increased  the  clinical 
confidence"  of  the  providers.  It  is  interesting  to  note,  however, 
that  in  each  X-ray  category,  some  radiologists'  reports  were  reported 
to  have  had  a  major  effect  on  care  (6%  for  routine  physicals;  8%  for 
emergency  exams;  10%  for  other  diagnostic  exams;  and  7%  for  exams 
performed  "for  the  record"  or  as  follow-up  procedures) . 

2.  User  Acceptance 

During  the  post-implementation  data  collection  period,  system 
users  were  questioned  regarding  their  opinions  of  the  system  and  its 
utility.  The  response  rate  for  each  survey  was  over  90%. 
a.  Acceptance  by  Primary  Care  Providers 

Overall,  providers'  comments  were  positive.  Most  of  their 
enthusiasm,  however,  was  derived  from  the  system's  potential  utility 
rather  than  actual  benefits  realized  during  the  field  trial. 
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Providers  believed  that  the  system  could  be  valuable  where  turnaround 
tine  reduced  to  a  few  hours  or  to  a  single  day.  Also,  at  some  of  the 
sites,  providers  felt  that  24-hour  availability  of  Interpretation 
services  was  Important. 

b.  Acceptance  by  Receiving  Site  Radiologists 

The  radiologists  who  had  participated  in  the  field  trial  at 
Halcolm  Grow  Medical  Center  completed  a  written  questionnaire 
regarding  their  experience  with  the  system.  The  radiologists' 
comments  were  enthusiastic.  They  felt  that  the  quality  of  images 
received  was  generally  good  and  that  image  resolution  was  usually 
adequate. 

c.  Acceptance  by  Technicians  and  System  Operators 

Each  technician  who  used  the  system  at  the  transmitter  sites  was 
Interviewed.  All  felt  that  once  the  system  had  become  reliable,  it 
had  been  easy  to  use.  This  opinion  was  expressed  both  by  trained 
radiology  technicians  and  by  the  non-technician  system  operators.  The 
non-technicians  did  require  somewhat  more  time  to  become  accustomed  to 
the  system  —  to  learn  the  correct  positioning  and  focusing  of  films 
--  but  soon  became  very  adept  at  its  operation. 

Transmitter  site  personnel  did  criticize  some  aspects  of  system 
design,  primarily  complaining  that  the  film  Inputting  activity  was 
tedious  and  time-consuming. 

3.  feasibility  of  Routine  Use 

In  order  to  determine  the  feasibility  of  using  the  Teleradiology 
system  in  routine  medical  practice  settings,  its  potential  impact  on 
the  dally  routines  was  studied  at  both  the  transmitter  sites  and  at 
the  central  receiver  site. 

Several  days  of  work  sampling  were  performed  at  each  of  the  four 
transmitter  sites  both  before  and  after  system  Implementation.  During 
both  study  periods,  approximately  fifty  percent  of  X-ray  department 
staff  time  was  spent  in  activities  unrelated  to  X-ray  department  work. 
(Workload  is  extremely  uneven  in  these  small  departments,  and  much  of 
this  time  was  spent  "on-call"  for  X-ray  duty  during  non-busy  times.) 
Performing  and  procesalng  each  X-ray  and  doing  the  paperwork  and 
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filing  associated  with  radiographs  took  between  20  minutes  and  AO 
minutes  per  exam;  inputting  Teleradiology  images  required  an 
additional  10  minutes  per  exam.  It  is  assumed  that  these  figures  are 
good  estimates  of  the  amount  of  time  that  would  be  required  were  the 
system  in  routine  use.  Although  all  but  one  of  the  transmitter  sites 
did  Increase  their  staff  between  our  first  and  second  study  periods, 
the  data  fail  to  show  that  these  staff  increases  would  be  required  to 
accommodate  routine  system  operation. 

Time  studies  performed  on  radiologists  interpreting  X-ray  films 
were  conducted  before  system  implementation  and  similar  studies  were 
conducted  under  Teleradiology.  It  was  determined  that  video  viewing 
is  only  slightly  more  time-consuming  than  film  viewing. 

From  these  work  sampling  and  time  study  data,  one  could  infer 
that  small  moderately  busy  X-ray  departments  should  be  able  to 
accommodate  Teleradiology  system  operation  into  their  daily  schedules 
without  an  increase  in  staff.  Large  radiology  departments  who 
currently  accommodate  interpretations  of  X-ray  films,  could  similarly 
accommodate  interpretations  of  the  Teleradiology  Images  of  the  films. 
A,  System  Costs 

To  determine  the  potential  impact  of  Teleradiology  on  X-ray 
department  costs,  the  estimated  costs  of  the  Teleradiology  system  at 
the  four  transmitter  clinics  were  compared  with  the  two  "manual" 
methods  for  obtaining  radiologists'  interpretations  of  X-rays 
performed  at  these  clinics.  These  two  "manual"  methods  are  (1)  using 
a  courier  to  transport  films  and  (2)  employing  a  part-time  visiting 
radiologist.  In  the  field  trial  transmitter  sites,  the  equipment  and 
staff  necessary  to  perform  and  process  X-ray  exams  are  essentially  the 
same  regardless  of  which  of  the  three  systems  is  used.  Hence 
attention  was  focused  only  on  the  incremental  costs  assc'iated  with 
the  three  alternatives.  The  cost  of  using  the  experimental 
Teleradiology  system  in  the  field  trial  sites  was  found  to  be 
approximately  $7  per  X-ray  exam,  compared  with  an  estimated  $2.50  per 
exam  when  a  part-time  visiting  radiologist  is  employed  and  $0.50  when 
a  courier  system  is  used.  These  relative  costs  would,  of  course,  vary 
in  different  settings. 
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D 


SUMMARY  AMD  CONCLUSIONS 


A  vefined  Teleradiology  system  would  make  available  to  remote 
clinics  the  same  access  to  radiologists'  interpretation  services  as  la 
currently  available  to  large  hospital  outpatient  departments.  The 
remote  clinics  would  have  routine  interpretations  returned  to 
providers  within  a  day  or  two  and  "wet  readings"  of  films  would  be 
accessible  at  any  time  during  the  day.  Primary  responsibility  for 
radiological  interpretation  would  be  shifted  from  the  primary  care 
provider  to  the  radiologist. 

At  clinics  typified  by  the  four  transmitter  sites  involved  in  the 
field  trial,  it  appears  that  Teleradiology  can  provide  a  much  more 
rapid  turnaround  of  radiologists'  interpretation  reports  than  can 
either  of  the  standard  "manual"  methods  for  obtaining  film 
interpretations.  The  system  appears  feasible  to  use  in  small 
transmitter  site  X-ray  departments  and  a  large  central  receiving  site. 
It  has  been  demonstrated  to  be  acceptable  to  users.  At  the  field 
trial  sites,  alternative  methods  for  obtaining  interpretations  were 
readily  available,  and  were  less  expensive  than  using  Teleradiology. 

In  the  majority  of  cases,  providers  in  transmitter  clinics 
expressed  a  preference  for  receiving  expert  interpretations  promptly 
rather  than  relying  on  their  own  readings  of  films.  However,  the 
importance  of  X-ray  examinations  and  the  significance  of  prompt 
receipt  of  radiologists'  interpretations  were  found  to  vary  with  X-ray 
exagt  type.  Providers  indicated  most  often  that  radiographic  findings 
were  relevant  to  immediate  care  in  trauma  cases,  less  often  that 
findings  were  critical  for  other  diagnostic  exams,  followed  by 
examinations  performed  "for  the  record"  or  for  follow-up,  and, 
finally,  they  were  least  often  felt  to  be  immediately  relevant  for 
routine  physical  examinations. 
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I.  INTRODUCTION 


This  report  presents  the  final  results  of  an  evaluation  conducted 
by  Arthur  D.  Little,  Inc.,  concerning  the  functional  performance  of 
the  Washington-area  Teleradiology  Field  Trial  System.  The  evaluation 
was  designed  to  assess  the  utility  of  teleradiology  in  medical 
practice. 

A.  BACKGROUND 

Many  small  health  care  facilities,  especially  those  located  in 
remote  areas,  do  not  have  a  full-time  staff  radiologist,  but  do 
provide  X-ray  services  to  their  patients.  Most  examinations  performed 
at  such  facilities  are,  nevertheless,  interpreted  by  a  radiologist. 
There  are  two  reasons  for  this  practice: 

•  the  specialist's  review  is  considered  important  for 
the  provision  of  good  quality  diagnostic  care;  and 

•  a  radiologist's  interpretation  is  recommended  by  the 
Joint  Commission  on  Accreditation  of  Hospitals. 

There  are  two  common  methods  for  accomplishing  radiologist’s 
review  of  exams  performed  at  small  X-ray  departments: 

•  the  films  are  transported  from  the  small  site  to  a 
radiologist's  workplace;  or 

•  a  radiologist  is  transported  to  the  small  site  on  a 
part-time  basis. 

Compared  with  the  situation  of  having  a  full-time  staff  radiologist, 
each  of  these  methods  of  obtaining  interpretations  has  limitations: 

•  a  significant  time  interval  often  exists  between 
the  request  for  an  X-ray  examination  and  when  a 
radiologist's  interpretation  is  available  for  use 

by  the  primary  care  provider  in  medical  decision-making; 

•  there  is  a  cost  associated  with  the  packaging  and 
transport  of  films,  the  transport  of  personnel  and 
patients,  and  non-productive  time  spent  by  personnel 

and  by  patients  in  transit;  and 

•  X-ray  films  and  reports  may  be  lost  in  transit. 
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B.  THE  TELERADIOLOCY  FIELD  TRIAL 


"Teleradiology"  is  an  automated  system  whereby  an  electronic 
representation  of  an  X-ray  image  is  transmitted  via  telephone  wires 
from  one  location  to  another.  The  system  allows  a  radiologist  at  one 
central  site  to  interpret  X-ray  examinations  from  several  small  or 
remote  sites.  Teleradiology  interpretation  can  occur  in  a  more  timely 
fashion  than  film  interpretation;  also  the  system  does  not  require  the 
transportation  of  films,  personnel  or  patients.  It  thus  has  the 
potential  for: 

•  increasing  access  to  radiologists'  Interpretation 
services  at  small  or  remote  facilities  by  reducing  the 
interval  between  the  time  when  an  X-ray  examination  is 
requested  and  the  time  when  a  radiologist  s 
interpretation  is  available  for  use  by  the  primary  care 
provider  in  medical  decision-making;  and 

•  reducing  the  dollar  costs  of  providing  X-ray  services 

at  small  or  remote  sites  by  centralizing  interpretation 
services  and  by  eliminating  the  costs  associated  with 
transportation,  with  non-productive  time  spent  by 
personnel  and  patients  in  transit  and  with  film  and 
report  losses. 

In  order  ;o  evaluate  the  utility  of  teleradiology  in  routine 
medical  practice,  a  teleradiology  system  (designed  by  the  MITRE 
Corporation)  was  installed  and  studied  in  the  Washington,  D.C.  area. 
This  field  trial  lasted  6  months.  Tt  was  undertaken  with  the  support 
of  the  Public  Health  Service  and  the  Tri-Service  Medical  Information 
System  (TRIMIS)  Program  Office.  Evaluation  activities  were  conducted 
by  the  Bureau  of  Radiological  Health  (Public  Health  Service) ,  by  the 
MITRE  Corporation  and  by  Arthur  D.  Little,  Inc. 

The  teleradiology  system  was  installed  in  January  1982,  became 
fully  operational  in  March,  and  remained  so  through  June.  It  allowed 
transmission  of  X-ray  images  from  each  of  four  small  transmitter  sites 
to  one  central  reading  site.  The  transmitter  sites  involved  in  the 
teleradiology  field  trial  were: 
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•  Bolling  Air  Force  Base  Clinic  in  Washington,  D.C.; 

•  Fort  Detrick  Army  Clinic  in  Frederick,  Maryland; 

•  Patuxent  Naval  Air  Station  Hospital  in  Lexington 
Park,  Maryland;  and 

•  Central  Virginia  Community  Health  Clinic  in 
New  Canton,  Virginia. 

The  central  reading  site  for  the  field  trial  was  Malcolm  Grow 
Medical  Center  on  Andrews  Air  Force  Base  in  Maryland. 

Because  the  teleradiology  system  was  experimental,  it  was  not 
used  as  a  replacement  for  normal  film  interpretation  of  X-ray  exams 
during  the  field  trial.  At  each  transmitter  site,  the  "manual" 
methods  for  obtaining  interpretations  that  had  been  used  prior  to 
teleradiology  continued  to  be  used  in  parallel  with  the  automated 
system. 

C.  EVALUATION  OBJECTIVES  , 

The  field  trial  of  teleradiology  had  two  goals: 

(1)  to  determine  if  teleradiology  allows  a  radiologist 
to  interpret  X-ray  images  accurately;  i.e.,  to 
discover  how  well  the  system  performs  technically;  and 

(2)  to  determine  whether,  given  a  certain  level  of 
technical  performance,  teleradiology  is  useful 

in  a  clinical  setting;  e.g.,  to  discover  how  well 
the  system  performs  functionally . 

The  technical  evaluation  (Goal  !)  was  conducted  by  the  MITRE 
Corporation  and  the  Bureau  of  Radiological  Health,  and  will  not  be 
specifically  discussed  here. 

The  functional  evaluation  (Goal  2)  was  conducted  by  Arthur  D. 
Little,  Inc.,  and  is  the  subject  of  this  report.  The  functional 
evaluation  was  conducted  as  a  modified  before-and-af ter  study:  one 
stage  of  data  collection  was  performed  before  the  teleradiologv  system 
was  implemented  (baseline  or  Period  X)  and  one  was  performed  while  the 
system  was  operating  in  parallel  with  manual  methods  (post¬ 
implementation  or  Period  Y).  In  addition,  Arthur  D.  Little,  Inc., 
staff  were  involved  in  monitoring  the  implementation  of  the  system. 
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A  summary  of  the  data  collected  by  Arthur  D.  Little,  Inc.,  is 
presented  in  the  following  chapters.  Also,  conclusions  concerning  the 
functional  utility  of  teleradiology  are  presented  in  this  report. 
These  were  drawn  from  a  comparison  of  information  collected  in  Periods 
X  and  Y  and  concern: 

•  the  extent  to  which  teleradiology  has  the  potential  to 
improve  patient  care  and  alter  patient  disposition; 

•  the  potential  feasibility  of  using  teleradiology  in  routine 
medical  practice; 

•  the  potential  acceptability  of  teleradiology  to  users;  and 

•  estimates  of  the  incremental  costs  of  using  teleradiology 
versus  the  manual  alternatives. 

D.  DATA  COLLECTION  ACTIVITIES 

1 .  Schedule 

Baseline  data  were  collected  through  the  entire  month  of  October 
and  the  first  weeks  of  November,  1981,  and  post-implementation  data 
were  collected  through  the  month  of  June  and  into  July,  1982.  The 
primary  data  collection  instruments,  the  X-ray  time  history  forms, 
were  used  to  track  two  weeks  of  X-rays  at  each  site  during  each  data 
collection  period.  A  schedule  of  data  collection  activities  is 
presented  in  Table  1,  and  the  data  collection  instruments  are  pre¬ 
sented  in  Appendix  A. 

2.  The  Data 

a.  Data  collection  Instruments  X-l  and  Y-l 

Time  History:  X-Ray  Request /Report  Cycle 

These  instruments  were  used  to  collect  descriptive  information 
regarding  the  X-ray  patients  seen  and  X-ray  examinations  performed 
during  the  study  periods.  Also,  opinions  of  referring  providers 
regarding  how  each  exam  was  used  In  patient  care  and  the  significance 
of  prompt  receipt  of  each  interpretation  report  were  gathered;  and  the 
time  that  various  events  in  the  request/report  cycle  occurred  was 
noted.  Separate  portions  of  this  form  were  completed  by  providers. 
X-ray  department  staff,  radiologists,  and  of  Arthur  D.  Little,  Inc., 
staff. 
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b. 


Data  collection  instruments  X-2  and  Y-2 


Cost  Data  Collection  Form 

One  copy  of  each  of  the  cost  data  collection  forms  was  used  by 
Arthur  D.  Little,  Inc.,  staff  at  each  site  to  record  the  elements  of 
the  cost  of  operating  the  transmitter  X-ray  departments.  These  data 
were  summed  and  extrapolated  to  produce  total  costs  per  year,  per 
patient,  and  per  examination,  and  to  estimate,  the.  costs  of  using 
teleradiology  versus  manual  methods  for  obtaining  X-ray 
interpretations. 

c.  Data  collection  instruments  X-3  and  Y-3 

Work  Analysis  for  Each  X-Ray  Technologist  and  Other  Relevant 

Personnel 

Arthur  D.  Little,  Inc.,  staff  used  these  forms  to  observe  and  to 
record  the  activities  of  X-ray  department  staff  at  regular  intervals, 
in  order  to  determine  the  distribution  of  staff  time  among  various 
functions. 

d.  Data  collection  instruments  X-4  and  Y-4 

Radiologist:  X-Ray  Interpretation  Time  Study 

These  forms  were  designed  to  provide  information  concerning  the 
amount  of  time  required  for  interpretation  of  X-ray  films  and  X-ray 
images  from  the  transmitter  sites.  During  the  baseline  period, 
radiologists  interpreting  X-ray  examinations  performed  at  the 
transmitter  sites  were  asked  to  complete  the  time  study  form,  while 
during  the  post-implementation  period,  Arthur  D.  Little,  Inc.,  staff 
completed  this  form  using  data  collected  by  the  system. 

e.  Data  collection  instrument  Y-5 

Provider  Questionnaire 

This  form  was  distributed  during  the  post-implementation  data 
collection  period  to  the  health  care  providers  who  order  X-rays  at  the 
transmitter  sites.  It  was  designed  to  gather  their  impressions  of  the 
teleradiology  system  and  their  overall  opinions  of  X-ray  department 
operations  at  their  facilities. 
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f.  Data  collection  Instrument  Y-6 
Radiologist  Questionnaire 

This  form  was  mailed  to  all  radiologists  who  had  interpreted 
teleradiology  images  at  the  central  receiver  site.  It  was  designed  to 
gather  radiologists'  impressions  of  the  system. 

g.  Data  collection  instrument  Y-7 
Interview  Guide 

This  form  was  used  as  a  guide  for  interviews  conducted  during  the 
post-implementation  period  with  all  system  operators  and  project 
officers  involved  in  the  field  trial  at  the  transmitter  sites.  The 
interviews  concerned  users'  impressions  of  the  system. 
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II.  SETTING 


The  central  reading  site  for  the  teleradiology  field  trial  was 
the  Malcolm  Grow  Medical  Center  (MGMC) .  Four  transmitter  sites  were 
involved  in  the  field  trial: 

•  Bolling  Air  Force  Base  Clinic  (Bolling); 

•  Fort  Detrick  Army  Clinic  (Detrick) ; 

•  Patuxent  Naval  Air  Station  Hospital  (Patuxent);  and 

•  Central  Virginia  Community  Health  Center  (CVCHC) . 

Images  of  the  X-ray  exams  performed  at  the  transmitter  sites  were 
interpreted  by  a  radiologist  situated  at  the  central  reading  site. 
His  interpretation  was  then  sent  back  to  the  appropriate  transmitter 
site  for  use  in  patient  management  and  care  there. 

This  chapter  provides  an  overview  of  the  characteristics  of  the 
teleradiology  system  and  its  operation,  followed  by  a  brief 
description  of  each  of  the  five  study  sites  and  their  X-ray 
departments.1  Characteristics  of  the  transmitter  sites  and  their 
X-ray  departments  are  summarized  in  Tables  2  and  3.  The  operations  of 
the  transmitter  departments  are  described  in  greater  detail  in 
Appendix  B  of  this  report. 

A.  THE  TELERADIOLOGY  SYSTEM 

1.  Transmitter  Units 

At  each  of  the  four  transmitter  sites  the  following  system 
components  were  installed: 

•  a  large,  horizontally  placed  lightbox; 

•  a  video  camera  and  zoom  lens  for  imagp  capture 
(located  several  feet  above  the  lightbox); 

•  a  14"  video  monitor  on  which  images  visible  through 
the  camera  could  be  viewed; 

•  a  512x512  frame  freeze  device  for  image  processing; 

•  a  CRT  screen  and  keyboard  for  entering  patient  and  image  data; 

^he  specific  system  components  are  also  listed  in  Tables  C-10  and 
C-ll  in  the  Appendix. 
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OF  TRANSMITTER  SITE  FACILITIES 


•  a  control  processor /convertor  for  processing  image  data; 

•  a  200  Megabyte  disk  for  data  storage; 

•  a  data  error  corrector; 

•  a  modem  for  information  transmission;  and 

•  a  word  processor  printer. 

These  components  occupied  a  vertical  rack  (approximately  5,x3'x2')  and 
the  surface  of  a  medium-sized  desk. 

To  use  the  system  at  a  transmitter  site,  the  system  operator: 

a.  entered  relevant  patient  data  via  the  keyboard  (in 
response  to  prompts  that  appeared  on  the  CRT  screen); 

b.  keyed  in  certain  commands; 

c.  placed  an  X-ray  film  on  the  lightbox; 

d.  adjusted  controls  on  the  canera  and  on  the  monitor 
until  the  image  appearing  on  the  monitor  screen  was 
clear  and  in  focus; 

e.  keyed  in  an  additional  code,  commanding  the  system 
to  capture  and  save  an  image  of  the  X-ray  film. 

f.  repeated  steps  b.  through  e.  until  all  relevant  images 
were  captured. 

The  process  of  inputting  the  images  for  one  exam  took  between  5  and  10 
minutes,  depending  upon  the  nature  of  the  exam  and  the  number  of  films 
to  be  input.  Radiologists'  interpretation  reports  were  later  received 
at  the  transmitter  site  via  the  word  processor  printer. 

2.  Receiver  Unit 

At  the  central  reading  site,  there  were: 

•  four  9600  baud  modems  and  four  disk  drives  for  receiving 
and  storing  data  from  the  transmitter  sites; 

•  a  data  error  corrector; 

•  a  control  processor  for  routing  information  through  the 
system; 

•  an  image  processing  and  display  system; 

•  a  CRT  and  keyboard; 

•  three  video  monitors  for  viewing  X-ray  images; 

•  a  word  processor;  and 

•  a  word  processor  printer. 
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The  first  three  components  occupied  three  vertical  racks  (each 
approximately  5,x3'x2’);  the  video  monitors  and  CRT  occupied  a  large 
desk  and  a  small  table.  The  word  processor  printer  was  placed  nearby 
the  CRT;  the  word  processor  was  located  in  a  separate  room. 

To  interpret  images  at  the  central  receiver  site,  the  radiologist 
entered  certain  commands  via  the  CRT  keyboard,  thereby  selecting  the 
images  that  he  wished  to  view  from  a  menu  list  appearing  on  the  CRT 
screen.  The  selected  images  appeared  on  the  three  viewing  monitors 
and  patient  data  that  had  been  entered  at  the  transmitter  site  were 
printed  at  the  word  processor  printer  and  were  available  for  the 
radiologist  to  use  in  interpretation. 

When  interpretation  had  been  completed  —  a  process  taking  about 
as  long  as  film  interpretation  —  the  radiologist  dictated  his 
interpretation  and  "called  up"  the  next  case.  The  dictated 
interpretation  was  transcribed  into  the  word  processor  by  a  typist  and 
transmitted  back  to  the  transmitter  site.  Cases  from  each  transmitter 
site  were  interpreted  in  the  order  of  input;  the  system  was  programmed 
to  queue  the  cases  depending  upon  the  number  of  cases  stored  on  disk 
from  a  given  site.  Approximately  6  weeks  before  the  end  of  the  trial, 
a  PRIORITY  mode  was  installed.  This  allowed  individual  cases  to 
circumvent  the  standard  queuing  process  once  the  central  site  operator 
had  been  alerted  by  telephone. 

B,  THE  CENTRAL  READING  SITE 

Malcolm  Grow  Medical  Center  (MGMC) ,  the  central  reading  site,  is 
a  280-bed  tertiary  medical  center,  with  a  large  outpatient  workload. 
It  Is  located  on  Andrews  Air  Force  Base  in  Maryland,  10  miles 
southeast  of  Washington,  D.C.  MGMC  has  a  large  radiology  department, 
staffed  by  five  or  six  radiologists  and  33  technicians.  Radiologists 
in  the  department  interpret  the  X-rays  from  Bolling  Air  Force  Base 
Clinic,  the  Pentagon,  and  the  Air  Force  Clinic  in  the  Azores  as  part 
of  the  daily  workload. 

During  the  field  trial  the  MGMC  radiology  department  continued, 
for  the  most  part,  to  operate  as  it  had  before.  The  teleradiology 
receiver  equipment  was  physically  separated  from  the  other  viewing 
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rooms  in  the  department;  the  transcriptionista  who  used  the  system 
were  especially  hired  for  the  project;  and  the  radiologists  who 
interpreted  teleradiology  images  were  recruited  from  several  hospitals 
in  the  area  and  used  the  system  on  a  part-time,  scheduled,  volunteer 
basis. 

C.  THE  TRANSMITTER  SITES 
1.  Bolling  Air  Force  Base  Clinic 

Bolling  Air  Force  Base  Clinic  (Bolling)  is  a  family  practice 
clinic  located  on  Bolling  Air  Force  Base  in  Washington,  D.C.  The 
clinic  provides  primary  care  services  to  Air  Force  active-duty 
personnel  and  their  dependents  who  reside  on  or  nearby  the  base  and 
has  approximately  52,000  outpatient  visits  each  year.  It  is  a 
satellite  facility  of  the  Malcolm  Grow  Medical  Center  at  Andrews  Air 
Force  Base,  located  10  miles  away,  and  relies  on  MGMC  for  secondary 
and  tertiary  care  referrals. 

The  X-ray  department  at  Bolling  is  staffed  by  one  or  two  X-ray 
technicians  and  approximately  3,400  patients  are  examined  there  each 
year.  All  X-ray  examinations  performed  at  the  clinic  are  sent  to  MGMC 
for  interpretation. 

During  the  first  half  of  the  field  trial  (from  January  to  March 
1982),  Bolling  was  located  in  temporary  quarters.  During  this  period 
the  clinic  provided  limited  clinical  services  and  no  X-ray  services  to 
its  patients.  Bolling  patients  who  required  X-rays  were  referred  to 
MGMC.  Bolling's  teleradiology  transmitter  equipment  was  temporarily 
installed  at  MCMC  and  films  taken  at  MGMC  of  Bolling  patients  were 
input  into  the  system  at  MGMC  by  MGMC  X-ray  technicians. 

During  the  latter  half  of  the  trial,  the  Bolling  clinic  did 
provide  X-ray  services.  X-rays  performed  at  Bolling  were  input  into 
the  teleradiology  transmitter  equipment  located  in  a  room  adjacent  to 
the  X-ray  department.  Most  inputting  was  performed  by  a  system 
operator,  specially  hired  for  the  project,  who  input  for  several  hours 
2  days  each  week.  On  a  few  occasions,  the  Bolling  technicians  input 
films  themselves. 
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2.  Fort  Detrlck  Army  Clinic 

The  Fort  Detrlck  Army  Clinic  (Detrick)  is  a  primary  care  clinic 
located  on  Fort  Detrick  Army  Base  within  the  facility  of  the  United 
States  Army  Medical  Research  Institute  for  Infectious  Diseases  in 
Frederick,  Maryland.  It  serves  the  large  community  of  retired 
military  personnel  and  their  dependents  in  the  area,  and  the  active- 
duty  personnel  and  their  dependents  who  are  stationed  at  the  several 
small  Army  bases  nearby.  The  clinic  has  approximately  21,000  out¬ 
patient  visits  each  year.  The  X-ray  department  and  laboratory  of  the 
clinic  also  serve  staff  associated  with  the  Litton  Bionetics  installa¬ 
tion  on  base,  and  perform  some  tests  related  to  the  research 
activities  in  the  building. 

Detrick  is  a  satellite  of  the  Walter  Reed  Army  Medical  Center 
(WRAMC)  and  relies  on  WRAMC  for  secondary  and  tertiary  care  back-up. 
The  clinic  is  physically  connected  to  a  10-bed  inpatient  ward,  which 
provides  limited  hospital  services.  This  inpatient  unit  is  available 
for  admissions  from  the  clinic  and  provides  24-hour  emergency  care  for 
eligible  patients.  It  is  primarily  used  for  research  activities 
unrelated  to  the  clinic  and  is  usually  staffed  by  physicians  who  do 
not  work  in  the  clinic. 

The  X-ray  department  at  Detrick  has  two  or  three  X-ray 
technicians  and  sees  approximately  4,000  patients  each  year.  All 
X-rays  performed  at  Detrick  are  interpreted  there  by  a  radiology 
resident  from  WRAMC,  who  works  at  the  clinic  2  days  each  week. 

During  the  field  trial  all  exams  performed  at  Detrick  were  input 
into  the  teleradiology  transmitter  equipment  located  in  the  X-ray  film 
viewing  room  there.  The  regular  staff  in  the  X-ray  department  was 
assisted  first  by  a  MITRE  staff  member,  who  input  films  into  the 
system  for  a  few  hours  2  days  each  week,  and  later  by  a  temporary 
full-time,  active-duty  X-ray  technician,  who  was  especially  assigned 
to  Detrick  for  the  latter  half  of  the  field  trial  (April  -  June  1982). 

3.  Patuxent  Naval  Air  Station  Hospital 

The  Patuxent  Naval  Air  Station  Hospital  (Patuxent)  is  located  on 
the  grounds  of  the  Naval  Air  Station  in  Patuxent  River,  Maryland.  It 
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is  the  only  hospital  in  the  immediate  area  and  provides  medical  care 
to  active-duty  personnel  and  their  dependents  stationed  at  the  Air 
Station.  This  hospital  is  primarily  an  outpatient  facility,  seeing 
85,000  outpatients  each  year,  but  also  has  20  inpatient  beds,  two 
surgical  suites,  and  a  24-hour  emergency  room.  Patients  requiring 
specialized  care  are  referred  to  the  National  Naval  Medical  Center 
(NNMC)  in  Bethesda,  90  miles  away. 

The  X-ray  department  at  the  hospital  has  four  X-ray  technicians 
and  sees  approximately  7,000  patients  each  year.  Every  other  week,  a 
radiology  resident  from  NNMC  works  at  the  hospital  from  Monday  through 
Thursday;  during  the  off  weeks,  X-ray  examinations  are  sent  by  courier 
three  times  each  week  to  NNMC  for  interpretation. 

During  the  field  trial,  all  exams  performed  at  Patuxent  were 
input  into  the  teleradiology  transmitter  equipment  located 
approximately  40  feet  from  the  X-ray  department,  in  a  room  adjacent  to 
the  hospital's  operating  room.  The  Patuxent  system  was  primarily 
operated  by  a  full-time  clerk  especially  hired  by  MITRE  for  the  field 
trial.  Regular  X-ray  department  staff  members  assisted  this  clerk 
most  afternoons,  and  input  some  films  themselves. 

4.  Central  Virginia  Community  Health  Center 

The  Central  Virginia  Community  Health  Center  (CVCHC)  is  a  Public 
Health  Service-sponsored  community  health  clinic  located  in  the  rural 
area  of  New  Canton,  Virginia,  approximately  30  miles  southwest  of 
Charlottesville.  It  provides  primary  care  and  dental  services  to  area 
residents;  25,000  outpatients  are  seen  there  each  year.  The  clinic  is 
not  formally  affiliated  with  any  other  health  care  institutions,  but 
maintains  close  ties  with  the  University  of  Virginia  Medical  Center 
(UVMC)  in  Charlottesville,  which  provides  several  specialized  services 
on  a  contractual  basis.  Patients  in  need  of  secondary  or  tertiary 
care  are  referred  to  one  of  several  hospitals  in  the  surrounding  area. 

The  X-ray  department  at  the  clinic  has  one  radiology  technician 
and  one  X-ray  room;  approximately  1,400  patients  are  examined  there 
each  year.  All  X-ray  examinations  performed  at  the  clinic  are  sent  to 
UVMC  for  interpretation. 
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During  the  field  trial,  the  technician  at  CVCHC  input  all  X-ray 
films  performed  there  into  the  CVCHC  transmitter  equipment  located  In 
the  X-ray  department. 
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III.  METHODOLOGY 


A.  THE  APPROACH 

Teleradiology  is  intended  to  improve  access  to  radiologists’ 
interpretation  services  at  small  or  remote  facilities,  thus  improving 
patient  care,  altering  patient  disposition,  and  reducing  the  costs  of 
providing  X-ray  services.  The  teleradiology  system  used  during  the 
Washington-area  field  trial  was  experimental,  and  was  used  for  a 
limited  period  of  time.  It  was  hence  used  in  parallel  with,  rather 
than  as  a  replacement  for,  manual  methods  for  obtaining  radiologists' 
interpretations.  Tn  this  context,  the  system  was  not  expected  to  have 
major  effects  on  patient  care,  on  patient  disposition  patterns,  or  on 
operating  costs  during  the  field  trial. 

The  methodology  used  for  the  functional  evaluation  took  account 
of  these  aspects  of  the  field  trial  project.  Instead  of  employing  a 
pure  before/after  study  design,  the  approach  used  was  exploratory. 
The  study  was  intended,  in  part,  to  measure  actual  impacts  that 
occurred  in  the  experimental  setting,  but  was  designed  primarily  to 
estimate  potential  impacts,  which  would  be  expected  to  occur  if  the 
system  were  in  routine  operation. 

It  was  determined  that  four  general  impact  areas  could  be  studied 
in  the  experimental  setting: 

•  the  extent  to  which  teleradiology  could  potentially 
alter  patient  care  or  alter  patient  disposition; 

•  the  potential  feasibility  of  using  teleradiology  in 
routine  medical  practice; 

•  the  acceptability  of  teleradiology  to  its  users;  and 

•  the  estimated  incremental  costs  of  using  teleradiology 
versus  the  manual  alternatives. 

Each  of  these  areas  was  addressed  in  the  course  of  the  functional 
evaluation  effort. 
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B.  THE  STUDY  QUESTIONS 

In  order  to  determine  the  impacts  of  teleradiology  on  the  four 
areas  enumerated  above,  the  following  study  questions  were  formulated: 

1.  Patient  Care  and  Disposition 

#  If  teleradiology  resulted  in  a  change  In  the  time  lapse 
between  X-ray  request  and  report  review,  would  this 
change  alter  patient  care  or  alter  patient  disposition 
at  the  transmitter  sites? 

•  If  technical  limitations  of  the  teleradiology  system 

resulted  in  X-ray  images  being  more  difficult  to 

2  3 

interpret  than  X-ray  films  ’  and,  consequently,  resulted 
in  a  reduction  in  the  accuracy  of  interpretation,  would 
such  a  reduction  affect  patient  care  or  disposition  at 
the  transmitter  sites? 

2.  Feasibility  of  Routine  Use 

Would  use  of  the  teleradiology  system  alter  the  amount  of 
transmitter  site  and  receiver  site  staff  time  spent  in  X-ray 
processing  functions? 

3.  Acceptability  to  Users 

Would  teleradiology  be  acceptable  to  system  users? 

A.  Costs 

What  would  be  the  incremental  costs  of  using  the  teleradiology 
system? 

C.  THE  EVALUATION  MEASURES  AND  TECHNIQUES 

Tn  order  to  answer  the  study  questions,  a  set  of  evaluation 
measures  and  data  collection  techniques  were  devised.  These  are 

2 

Data  gathered  in  an  experimental  evaluation  of  the  teleradiology 
system  indicated  that  the  accuracy  of  interpretation  varies  between 
films  and  video  readings  of  the  same  X-ray.  In  both  interpretation 
modes,  accuracy  diminished  with  increased  'difficulty'  of  the  case  and 
the  difference  between  the  levels  of  accuracy  of  the  two  modes  also 
increased  with  increased  difficulty.  In  addition,  certain  types  of 
examinations  were  associated  with  lower  levels  of  accuracy  and  greater 
inter-modal  differences  in  accuracy  "scores."  In  essence,  for  simpler 
cases  and  for  more  routine  types  of  exams,  the  system  appeared  to 
perform  better  than  it  did  for  complex  or  difficult  cases. 

3 

Harrington,  M. ,  e£  al.  A  Laboratory  Evaluation  of  the  Teleradiology 
System:  Summer,  1980:  MITRE  Corporation,  1981,  Tables  3.1.2  and 
3.1.3. 
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summarized  in  Table  4.  The  data  collection  instruments  used  are 
contained  in  Appendix  A.  In  addition  to  specific  evaluation  measures, 
information  was  collected  regarding  two  independent  variables. 

1.  Data  were  collected  for  each  examination  during  the  study  period 
concerning  patient  and  examination  characteristics.  After  being 
distilled  into  four  X-ray  categories  (described  later) ,  these 
were  used  as  the  raa)or  independent  variable  in  analysis  of 
potential  impacts  related  to  patient  care  and  disposition. 

2.  Data  were  collected  for  each  examination  concerning  the  time 

lapses  in  various  stages  of  the  X-ray  request/report  cycle. 

The  extent  of  teleradiology's  impact  on  patient  care 

depends  in  large  part  on  its  impact  on  the  timeliness  of 
4 

this  cycle. 

D.  THE  CATEGORIES  OF  X-RAY  EXAMINATIONS 

In  order  to  group  the  X-ray  examinations  performed  during  the 
study  period  in  a  way  that  would  have  clinical  meaning  to  primary  care 
providers  and  to  radiologists,  the  following  X-ray  categories  were 
defined  and  used  in  analysis  of  data  related  to  impacts  on  patient 
care  and  disposition: 

•  Routine  Physical  chest  exams 

•  Emergency  exams 

•  (Non-Emergency)  Diagnostic  exams 

•  Exams  performed  For-the-record  or  as  Follow-up  procedures. 
These  categories  were  designed  to  group  X-rays  on  the  basis  of 

the  following  characteristics: 

•  the  likelihood  that  the  X-ray  examination  is  being  used 
as  the  primary  diagnostic  tool  in  immediate  patient  care 
and  disposition  decisions; 


4 

The  Bureau  of  Radiological  Health  collected  a  large  amount  of  data 
concerning  the  time  intervals  between  various  stages  in  the 
request /report  cycle  at  the  remote  sites  in  the  baseline  period. 
Some  additional  time  interval  data  were  collected  by  Arthur  D. 
Little,  Tnc.,  in  the  baseline  period.  Arthur  D.  Little,  Tnc. ,  was 
responsible  for  all  data  collection  in  the  post-implementation 
period. 
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•  the  level  of  skill  required  to  interpret  the  examination 
with  sufficient  accuracy  for  use  in  immediate  patient 
care  and  disposition  decisions; 

•  the  likelihood  that  the  examination  is  performed  with 
a  clear  expectation  that  results  will  be  normal  or 
abnormal. 

For  example,  chest  films  are  considered  difficult  for  a 
nonradiologist  physician  to  interpret.  However,  Routine  Physical 
chest  exams  seldom  have  an  immediate  effect  on  clinical  care:  their 
results  are  usually  expected  to  be  normal  and  the  X-ray  constitutes 
but  one  in  a  battery  of  diagnostic  tests. 

On  the  other  hand,  the  results  of  Emergency  exams  —  for  example, 
an  X-ray  performed  to  determine  whether  or  not  an  arm  is  fractured  — 
are  often  expected  to  be  abnormal  and  immediate  action  is  taken, 
depending  upon  whether  and  where  the  X-ray  indicates  fracture. 
Although  Emergency  films  certainly  require  experience  to  interpret 
consistently  and  accurately,  nonradiologists '  readings  may  be 
sufficient  for  determining  immediate  treatment. 

Non-emergency  Diagnostic  exams  may  involve  suspected  disorders  of 
greater  complexity  than  do  either  Emergency  exams  or  Routine  Physical 
chest  exams,  and  may  require  a  greater  degree  of  skill  for  adequate 
interpretation.  Such  conditions,  however,  are  less  likely  to  require 
immediate  treatment  than  are  cases  of  trauma. 

Finally,  exams  that  are  performed  For-the-record  or  as  Follow-up 
procedures  vary  in  the  amount  of  skill  required  for  their  adequate 
interpretation  and  whether  normal  or  abnormal  results  are  anticipated. 
However,  immediate  patient  care  decisions  are  unlikely  to  rest  solely 
on  their  results. 
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IV.  RESULTS 


A.  THE  INDEPENDENT  VARIABLES 
1.  Characteristics  of  X-ray  Volume 

The  value  of  prompt  X-ray  interpretation  report  turnaround 
depends  upon  the  case  mix  of  the  X-ray  patients  being  examined.  As  a 
measure  of  case  mix,  data  regarding  the  characteristics  of  patients 
and  examinations  were  collected. 

The  characteristics  of  the  X-ray  patients  examined  at  the 
transmitter  sites  during  the  two  study  periods  are  presented  in  Table 
5."*  Although  there  were  minor  variations  among  the  ages  of  the 
patients  seen  at  the  three  military  facilities,  the  great  majority 
were  between  the  ages  of  14  and  45,  with  older  patients  comprising  7% 
and  16%  of  the  populations  at  Bolling  and  Patuxent,  respectively,  and 
26%  of  the  X-ray  patients  at  Detrick.  At  CVCHC,  however,  66%  of  the 
patients  were  over  45  and  34%  were  over  64.  As  might  be  expected,  a 
majority  of  the  patients  at  the  military  facilities  were  male  (59%  to 
67%);  while  at  CVCHC,  the  X-ray  patients  were  nearly  equally 
distributed  by  sex. 

The  distribution  of  types  of  X-ray  examinations  performed  during 
the  two  study  periods,  the  reasons  for  the  examinations,  and  whether 
examinations  were  associated  with  trauma  are  presented  in  Table  6  for 
each  of  the  four  transmitter  sites.  At  all  of  the  sites,  approxi¬ 
mately  80%  of  examinations  were  either  extremity  or  chest  films. 
Also,  more  X-rays  were  considered  "diagnostic,"  rather  than  being 
performed  "for-the-record ,"  as  "follow-up"  procedures  or  in  associa¬ 
tion  with  routine  physical  examinations.  The  proportion  of  exams  per¬ 
formed  at  the  military  sites  "for-the-record"  and  in  association  with 
routine  physical  exams  was  nearly  equal  (23%  and  25%,  respectively). 


^Tables  C-l  and  C-2  in  the  Appendix  present  data  for  each  of  the  study 
periods  separately. 

^Tables  C-3  and  C-4  in  the  Appendix  present  data  for  each  of  the  study 
periods  separately. 
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Tables  C-l  and  C-2  in  the  Appendix  present  data  for  each  of  the  study  periods 
separately. 
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- — - - — — — -  '  . ,  ^  for  each  of  the  study  periods  separately 

‘Tables  C-3  and  C-4  In  the  Appendix  present  data  for  each 


while  many  fewer  of  the  exams  performed  at  CVCHC  fell  into  these 
categories.  Although  a  large  proportion  of  examinations  at  each  site 
were  associated  with  trauma,  the  percentage  of  trauma-re] ated  X-rays 
varied  somewhat  among  sites,  from  31%  at  CVCHC  to  43%  at  Patuxent. 

The  distribution  of  X-rays  performed  at  the  transmitter  sites 
during  the  study  period  are  presented  by  X-ray  category  in  Table  ij 
Bolling  and  Patuxent,  with  rather  young  patient  populations,  had  high 
proportions  (46%  and  43%,  respectively)  of  Emergency  exams,  compared 
with  Detrick  and  CVCHC  (each  at  28%).  CVCHC,  the  only  civilian  site 
involved  in  the  study,  had  the  smallest  proportion  of  Routine  Physical 
examinations  (6%,  compared  with  28%,  16%  and  25%  at  the  three  military 
sites) . 

The  X-ray  categories  tend  to  be  associated  with  certain  ages  and 

g 

sexes  of  patients  as  shown  in  Table  8.  A  higher  proportion  of  X-rays 
performed  on  adult  patients,  aged  14-45  and  46-64,  were  associated 
with  Routine  Physical  examinations  than  were  X-rays  of  younger  (under 
14)  and  older  (65+)  patients.  The  younger  age  groups  (under  14  and 
14-45)  had  higher  proportions  of  Emergency  exams  than  did  older 
patients  (46-64  and  65+) .  Older  patients  were  more  likely  to  have 
non-emergency  Diagnostic  exams  than  were  younger  ones;  and  the  exams 
of  older  patients  were  more  likely  to  be  performed  For-the-record  or 
as  Follow-up  procedures. 

Males  had  a  higher  proportion  of  Routine  Physical  exams  than  did 
females  (27%  compared  with  11%).  Females  had  proportionately  higher 
numbers  of  Diagnostic  exams  (33%  compared  with  males  at  15%).  The  two 
sexes  experienced  nearly  equal  proportions  of  Emergency  exams  and 
those  performed  For-the-record  or  as  Follow-up  procedures. 

In  summary,  the  characteristics  of  the  patients  seen  and 
examinations  performed  at  these  X-ray  departments  are  typical  of  X-ray 
case  mix  found  in  a  primary  care  setting.  In  the  military  sites. 


^Tables  C-5  and  C-6  in  the  Appendix  present  data  for  each  of  the  study 
periods  separately. 

g 

Tables  C-7  and  C-8  in  the  Appendix  present  data  for  each  of  the  study 
periods  separately. 
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TABLE  8 


X-ray  patients  tended  to  be  rather  young  —  84%  were  under  46  years 
old  —  and  rather  healthy  —  39%  of  exams  were  associated  with  trauma, 
23%,  with  routine  physical  examinations. 

2.  Elapsed  Time  in  the  X-Ray  Request/Report  Cycle 
a.  Turnaround  time 

The  major  potential  benefit  of  teleradiology  for  patient  care  is 
its  ability  to  reduce  turnaround  time  for  X-ray  interpretation 
reports.  To  measure  turnaround  time  and  its  components,  data  were 
collected  regarding  the  timing  of  the  various  stages  in  the  X-ray 
request /report  cycle. 

Data  regarding  mean  elapsed  times  in  various  stages  of  the  X-ray 
request /report  cycle  are  presented  by  facility  in  Table  C-9  in 
Appendix  C.  Mean  time  lapses  are  presented  there  for  stages  of  the 
X-ray  film  cycle  both  before  teleradiology  was  installed  and  during 
the  system's  operation,  and  for  the  X-ray  video  image  cycle  during  the 
post-implementation  period. 

Mean  elapsed  time  varied  substantially  between  facilities  and 
within  each  facility  at  each  stage  of  the  cycle,  during  each  data 
collection  period,  and  using  each  interpretation  mode.  The  mean  total 
time  required  for  the  X-ray  f ilm  request /report  cycle  ranged  from  88 
(+/-  33)  to  108  (  +  /-  44)  hours  at  the  four  transmitter  sites  before 
system  installation,  and  from  120  (+/-  75)  to  232  (+/-  111)  hours 
during  system  operation.  The  mean  total  request/report  cycle  for 
video  images  varied  From  99  (+/-  56)  hours  to  212  (+/-  104)  hours  at 
the  four  transmitter  sites. 

In  each  facility  and  using  each  viewing  mode,  exams  were 
performed  promptly  after  they  were  requested.  The  mean  time  lapse 
between  the  provider's  X-ray's  request  and  the  exam's  completion  was 
1.3  (+/-  5.6)  hours  overall.  Patients  in  the  transmitter  sites  —  all 
primary  care  facilities  —  usually  went  directly  from  their  provider 
encounter  to  the  X-ray  department  and  were  examined  there  almost 
immediately. 
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The  long  delays  of  the  request/report  cycle  occurred  after  the 
X-ray  examination  was  completed.  During  the  post-implementation 
period ,  mean  elapsed  time  between  exam  completion  and  exam 
interpretation  was  78  (+/-  96)  hours  overall  for  films  and  55  (+/-  68) 
hours  for  video  images;  and  that  between  exam  interpretation  and 
provider  report  review  was  119  (+/-  86)  hours  for  films  and  110  (+/- 
80)  hours  for  video  images. 

Although  in  a  nonexperimental  situation,  it  is  hypothesized  that 
teleradiology  interpretations  would  be  available  to  providers  more 
promptly  than  are  film  interpretations,  during  the  field  trial  the 
total  mean  turnaround  times  were  approximately  the  same  length  for 
image  interpretations  as  they  were  for  film  interpretations.  The  long 
delays  for  teleradiology  interpretations  experienced  during  the  field 
trial  were  largely  attributable  to  the  experimental  nature  of  the 
system  and  its  use: 

•  the  system  did  not  always  function  reliably; 

•  inputting  of  films  did  not  always  occur  regularly 
or  on  a  daily  basis; 

•  radiologists  were  only  available  to  perform  teleradiology 
interpretations  on  a  part-time  basis;  and 

•  "manual"  interpretation  reports  were  often  received  prior 

to  teleradiology  interpretations  of  the  same  exam,  and, 

9 

hence,  were  not  always  read  promptly. 

These  facts  about  the  experimental  situation  suggest  that  the  data 
collected  for  actual  teleradiology  interpretation  turnaround  time  do 
not  accurately  represent  what  a  routinely  operational  teleradiology 
system  would  be  like.  Indeed,  exams  can  be  input  into  the  system 
directly  after  completion;  inputting  takes  about  10  minutes  per  exam, 
and  image  transmission,  an  additional  15  to  30  minutes. 
Interpretation  can  occur  at  any  time  thereafter,  and  can  be 
communicated  immediately  back  to  the  transmitter  site. 


Overall,  for  41%  of  exams,  the  referring  provider  reviewed  the 
teleradiology  interpretation  before  the  film  interpretation  of  the 
same  exams;  for  another  41%,  the  film  interpretation  was  reviewed 
first;  and  for  18%,  the  two  interpretations  were  reviewed  at  the  same 
time. 
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The  field  trial  figures  demonstrate,  however,  that  installation 
of  the  teleradiology  technology  does  not,  itself,  result  in  reduced 
turnaround  time.  Although  system  operation  is  neither  extremely 
time-consuming  nor  very  complex,  for  the  system  to  be  used 
effectively,  protocols  must  be  established  for  inputting  films 
regularly,  radiologists  must  be  available  to  interpret  images  on  a 
routine  basis,  and,  as  under  any  reporting  system,  if  one's  goal  is  to 
minimize  time  delays,  reports  must  be  delivered  to  providers  promptly 
and  read  by  providers  upon  receipt. 
b.  Oral  Reporting 

Oral  reporting  can  substantial ly  reduce  total  turnaround  time  if 
it  occurs  shortly  after  interpretation,  and  it  was  presumed  at  the 
onset  of  data  collection  that  oral  interpretation  reporting  by 
radiologists  might  occur  frequently.  Hence,  the  incidence  of  oral 
reporting  of  radiologists'  interpretations  was  measured  during  both 
data  collection  periods. 

Only  12  instances  of  oral  communication  were  noted  for  the  418 
patients  in  the  baseline  study  sample  and  13  instances  occurred  for 
the  695  patients  in  the  post-implementation  sample.  It  can  be 

concluded  that  the  radiologist's  written  interpretation  report  is, 
usually,  the  only  radiologist's  interpretation  that  is  used  in  patient 
care  at  the  transmitter  sites. 

B.  PATIENT  CARE  AND  DISPOSITION 

1.  The  Role  of  the  X-Ray  Examination  in  Patient  Care  and 

Disposition  Decisions 

To  determine  how  X-ray  exams  and  interpretation  reports  were  used 
in  patient  care  at  the  transmitter  sites,  data  were  collected 

regarding  whether  referring  providers  viewed  the  exams  that  they 

ordered,  how  this  viewing  and  the  radiologist's  interpretation  report 
affected  treatment  decisions,  and  concerning  the  disposition  patterns 
of  X-ray  patients. 

a.  Does  the  referring  provider  view  the  X-ray  films? 

Tabulations  of  survey  responses  indicate  that  providers  who 
order  X-rays  often  view  the  films  themselves.  This  is  not  surprising, 
as  radiologists'  interpretations  are  usually  not  available  until 
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several  days  after  the  X-ray  exam  is  performed.  If  an  exam  is  to  be 
used  by  providers  for  immediate  diagnosis,  they  have  to  read  the  films 
themselves.  Table  9  shows  the  proportion  of  each  category  of  X-ray 
that  was  viewed  immediately  after  exam  performance.  Providers  were 
most  likely  to  view  films  for  Emergency  exams  (77%),  followed  by  other 
exams  that  they  considered  Diagnostic  (69%).  They  viewed  many 
For-the-record  and  Follow-up  examinations  as  well  (62%),  but,  not 
surprisingly,  relatively  few  Routine  Physical  chest  examinations 
(10%) . 

b.  What  effect  does  the  referring  provider's  viewing  of  the 

X-ray  films  have  on  his  handling  of  the  case? 

As  is  demonstrated  in  Table  10,  when  films  were  viewed  by 
referring  providers,  this  primarily  served  to  increase  clinical 
confidence.  This  was  the  case  for  65%  of  cases  and  is  to  be  expected: 
most  X-rays  are  performed  to  confirm  diagnoses  suspected  upon  physical 
examination.  For  15%  of  total  cases —  primarily  for  trauma  cases  or 
for  other  Diagnostic  exams  —  the  film  viewing  had  a  major  effect  on 
the  provider's  handling  of  the  case,  providing  really  new  information. 
It  was  least  likely  to  have  bad  any  effect  for  Routine  Physicals. 
c_.  What  effect  does  the  radiologist's  interpretation  report  have  on 

the  referring  provider's  handling  of  the  case? 

In  considering  providers'  responses  to  the  question,  "How  did  the 
radiologist's  report  affect  vour  handling  of  this  case?"  (Table  11), 
it  is  important  to  remember  that  this  report  is  almost  always  received 
several  days  after  the  X-rav  has  been  performed  and  the  patient 
treated  and  sent  home.  Reports  received  after  so  long  a  delay  might 
not  have  an  effect  on  patient  care  unless  their  findings  differed 
substantially  from  those  made  earlier  by  the  referring  provider. 

Overall,  42%  of  reports  were  considered  to  have  had  "No  effect" 
on  patient  care,  43%  were  felt  to  have  increased  the  primary  care 
provider's  clinical  confidence,  7%  to  have  altered  his  clinical 
impression,  and  8%  were  felt  either  to  have  had  a  major  effect  on 
patient  care  or  to  have  resulted  in  patient  recall. 
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PERCENTAGE  OF  EXAMINATIONS  VIEWED  BY  REFERRING  PROVIDERS 
BY  CATEGORY  OF  EXAMINATION3 
TELERADIOLOGY  FIELD  TRIAL 
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V 


A  different  data  base  was  used  for  this  analysis  than  for  Table  12;  hence,  these  data 
should  be  considered  separately. 


TABLE  10 

OPINIONS  OF  REFERRING  PROVIDERS: 
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As  might  have  been  expected,  interpretation  reports  concerning 
Routine  Physical  chest  exams  were  more  frequently  regarded  as  having 
had  "No  effect"  (54%)  than  reports  concerning  examinations  from  other 
categories. 

For  Emergency  exams,  interpretation  reports  were  not  often  felt 
to  have  significantly  affected  care:  only  8%  were  thought  to  have 
altered  clinical  decisions  and  5%  to  have  altered  clinical  impres¬ 
sions.  These  X-rays  are  likely  to  be  performed  for  immediate  use  in 
diagnosing  and  treating  fractures  —  77%  of  those  in  this  study  were 
viewed  by  providers  directly  after  they  were  performed  —  and  subse¬ 
quent  interpretations  were  not  of  great  value  in  handling  of  the  case. 

For  Diagnostic  exams  and  those  performed  For-the-record  or  for 
Follow-up,  relatively  high  proportions  of  radiologists'  reports  caused 
providers  to  alter  either  their  opinions  or  their  decisions  regarding 
patient  care.  The  percentage  of  reports  that  were  felt  to  have  had 
such  an  effect  were  20%  and  22%,  respectively.  These  exams  may  be 
somewhat  more  difficult  to  interpret  than  emergency  films,  and  the 
symptoms  that  call  for  their  performance  do  not  always  result  in 
immediate  treatment.  Radiologists'  interpretations  of  these 
examinations  would,  therefore,  be  more  likely  to  give  providers  new 
information  that  is  useful  in  patient  treatment  decisions.  Also,  the 
fact  that  providers  review  these  films  somewhat  less  often  than  they 
do  Emergency  exams  (69%  and  62%,  respectively,  versus  77%  for 
emergencies),  probably  enhances  the  value  of  the  radiologists'  reports 
in  the  handling  of  these  cases. 

It  is  important  to  notice,  however,  that  for  each  category  of 
examination,  some  interpretation  reports  did  have  a  major  effect  on 
care . 

d.  What  are  the  disposition  patterns  of  X-ray  patients? 

Table  12  displays  the  disposition  pattern  of  X-ray  patients 
examined  at  the  transmitter  sites.  The  data  from  the  baseline  and  the 
post-implementation  periods  were  pooled  for  presentation,  as  little 
variation  existed  in  patterns  of  patient  disposition  during  the  two 
study  periods.  Data  are  presented  regarding  patient  disposition  at^ 
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the  time  of  X-ray  request  and  at_  the  time  of  film  review  by  the 
referring  provider. 

Most  (66%)  of  patients  waited  in  the  clinics  for  their  providers 
(or,  in  a  few  cases,  for  a  radiologist)  to  interpret  their  X-rays. 
Not  surprisingly,  this  was  most  often  the  case  for  patients  with 
either  Emergency  exams  or  other  Diagnostic  exams  (88%  and  84%, 
respectively)  and  least  often  the  case  for  patients  undergoing  Routine 
Physical  examinations  (2%).  Overall,  66%  of  X-ray  patients  returned 
to  normal  activities  on  the  day  of  examination,  23%  were  told  to 
restrict  their  activities  and  the  remaining  11%  were  either  referred 
elsewhere  or  admitted  for  treatment  or  observation.  Of  patients  whose 
films  were  reviewed  directly  after  exam  performance,  50%  were  told  to 
return  to  normal  activities,  44%  to  restrict  their  activities,  3%  to 
go  to  another  medical  facility  for  further  tests  or  treatment  and  3% 
were  admitted  to  the  small  hospital. 

2.  Significance  of  Prompt  Receipt  of  the  Radiologist's 

Interpretation 

Providers  were  asked  to  indicate  for  each  exam  how  significant 
prompt  receipt  of  a  radiologist's  interpretation  would  be  to  patient 
care.  This  question  was  asked  twice  —  first,  at  the  time  that  he 
requested  the  X-ray,  and  second  at  the  time  that  he  received  the 
radiologist's  interpretation  report.  Data  relating  to  the  signifi¬ 

cance  of  prompt  receipt  of  the  X-ray  interpretation  are  presented  in 
Tables  13  and  14.  In  reviewing  these  data,  it  is  Important  to  keep 
two  facts  in  mind.  First,  the  referring  provider  usually  reviews  the 
films  himself  shortly  after  the  examination  has  been  performed  and 
must,  at  that  time,  make  a  decision  regarding  immediate  disposition  of 
the  patient.  Consequently,  the  provider's  first  opinion  concerning 

the  value  of  a  prompt  interpretation  most  likely  reflects  (a)  his  view 
of  the  diagnostic  function  of  the  X-ray  and  (b)  his  confidence  in  his 
own  ability  to  read  the  films.  Second,  the  referring  provider 
generally  receives  the  radiologist's  interpretation  several  days  after 
the  examination  has  been  performed  and  initial  patient  disposition 
made.  His  second  opinion  of  the  relative  value  of  prompt  interpreta¬ 
tion  receipt,  therefore,  probably  reflects  the  extent  to  which  the 

radiologist’s  interpretation  varies  from  his  own  reading  of  the  films. 
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Overall,  referring  providers  showed  much  more  interest  in  prompt 
interpretation  receipt  when  requesting  examinations  than  when  they 
reviewed  the  interpretation  reports.  At  the  time  of  X-ray  request, 
providers  felt  that  a  prompt  interpretation  would  significantly  (or 
very  significantly)  affect  clinical  handling  in  37%  of  cases. 
However,  at  the  time  of  report  review,  providers  believed  quite 
uniformly  that  a  more  timely  interpretation  would  not  have  altered 
care  (93%  of  cases,  overall).  In  other  words,  although  primary  care 
providers  frequently  wished  at  the  time  of  X-ray  request  that  a 
radiologist  were  available  to  interpret  films  promptly,  seldom  did  the 
radiologist's  report  contain  information  that  would  have  changed 
patient  care  had  it  been  received  earlier. 

a.  Significance  Considered  at  the  Time  of  the  X-Ray  Request 

Among  the  various  X-ray  categories,  at  the  time  of  X-ray  request, 
providers  indicated  least  often  that  a  prompt  interpretation  would  be 
significant  for  Routine  Physical  chest  examinations.  For  59%  of  such 
examinations,  a  prompt  interpretation  was  rated  as  "Not  significant." 
For  For-the-record  and  Follow-up  examinations,  a  prompt  interpretation 
was  frequently  considered  "Not  significant"  (in  29%  of  cases).  These 
results  are  consistent  with  the  premise  that  these  types  of 
examinations  tend  not  to  be  performed  for  use  in  immediate  therapeutic 
or  diagnostic  decisions. 

For  half  of  the  Emergency  exams,  a  prompt  interpretation  was 
rated  at  the  time  of  X-ray  request  as  either  "Not  significant"  or 
"Unlikely  to  affect  patient's  treatment  or  immediate  disposition." 
However,  in  the  majority  of  these  cases,  providers  believed  that  a 
timely  radiologist's  reading  would  serve  to  enhance  their  clinical 
confidence.  For  many  trauma-related  X-rays,  immediate  therapeutic 
action  is  apparently  taken  with  reasonable  confidence,  even  when  a 
radiologist's  interpretation  is  not  available. 

The  confident  and  accurate  interpretation  of  non-emergency 
Diagnostic  exams  generally  requires  a  greater  degree  of  X-ray  reading 
skill  than  do  trauma  examinations.  As  might  be  expected,  providers 
rated  a  prompt  interpretation  of  such  films  as  "Significant"  or  "Very 
significant"  in  62%  of  cases. 
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b.  Significance  Considered  at  the  Time  of  Report  Review 

Although  providers  felt  that  a  number  of  radiologists'  reports 
did  affect  the  handling  of  their  cases  (see  Table  11),  seldom  was  the 
timeliness  of  report  receipt  considered  significant.  Timely  receipt 
was  considered  important  for  7%  of  examinations,  overall.  It  is 
interesting  to  note  that  in  the  baseline  period,  providers  were  more 
likely  to  rate  prompt  receipt  as  significant  than  they  were  in  the 
post-implementation  period.  It  may  be  that  before  the  teleradiology 
system  was  installed,  when  attention  was  focused  on  the  system's 
future  arrival  and  accompanying  benefits,  turnaround  time  was  more  of 
an  issue  than  it  was  later,  during  relatively  routine  system 
operation. 

3.  Effects  of  Interpretation  Accuracy 

Interpretation  "Difficulty"  of  X-ray  Exams  performed 
If  technical  limitations  of  the  teleradioiogy  system  resulted  in 
X-ray  images  being  more  difficult  to  read  than  X-ray  films  and, 
consequently,  reduced  the  accuracy  of  interpretation,  would  such 
a  reduction  affect  patient  care  or  disposition  at  the  transmitter 
sites? 

Data  gathered  in  an  experimental  evaluation  of  the  teleradioiogy 
system  indicate  that  radiologists  do  interpret  teleradioiogy  video 
images  at  high  levels  of  accuracy.  However,  interpretation  accuracy 
does  vary  slightly  between  film  and  video  readings  of  the  same 
examination.  The  system  was  shown  to  perform  better  for  simpler, 
rather  than  more  difficult,  cases.  It  performed  best  for  extremity 
exams  (3.68  on  a  4-point  "accuracy  score"  scale  followed  by  chest 


Harrington,  M.  et  al.,  0£.  cit . 

The  4-point  scale  has  a  range  from  l="abnormality  not  noted"  to  4  = 
"abnormality  fully  characterized."  It  is  interesting  to  note  that 
differences  between  accuracy  scores  for  film  and  video  readings  of 
the  same  examinations  increased  in  parallel  with  decreasing  accuracy 
scores  for  video  readings;  i.e.,  the  more  difficult  it  was  for  a 
radiologist  to  read  a  given  film,  the  relatively  worse  his  video 
reading  of  that  same  examination  would  be.  The  differences  between 
accuracy  scores  for  films  and  video  readings  of  the  same  examination 
were  .16  (on  the  4-point  scale)  for  extremity  films,  .34  for  chest 
films,  .44  for  skull  films,  .46  for  abdomen  films,  and  .57  for  other 
skeletal  films. 
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exams  (3.44),  abdomen  exams  (3.28),  skull  exams  (3.08),  and  other 
skeletal  exams  (2.93). 

As  Is  apparent  from  data  presented  in  Section  1  of  this  chapter, 
a  substantial  portion  of  the  X-rays  performed  at  the  four  transmitter 
sites  involved  in  the  teleradiology  field  trial  is  made  up  of 
cases  of  low  complexity,  and  the  X-ray  department  workloads  involve 
high  numbers  of  the  types  of  examinations  for  which  the  system  has 
been  shown  to  perform  best.  At  all  sites,  over  70%  of  examinations 
were  either  extremity  or  chest  exams  (see  Table  6)  —  for  which 
teleradiology  has  been  shown  to  have  relatively  high  "accuracy 
scores."  More  "difficult"  abdomen  exams  make  up  6%  of  total 
examinations,  skull  exams  8%,  and  spine  exams  (other  skeletal),  6%. 

Therefore,  if  technical  limitations  in  the  teleradiology  system 
did  result  in  X-ray  images  being  more  difficult  to  read  than  X-ray 
films,  the  possible  reduction  in  interpretation  accuracy  may  have  less 
impact  in  these  primary  care  settings  than  it  would  in  facilities  with 
a  greater  number  of  complex  cases  or  more  "difficult"  examinations. 

C.  FEASIBILITY  OF  ROUTINE  USE 

1.  Transmitter  Sites 

In  order  to  determine  whether  the  teleradiology  system  could  be 
accommodated  into  normal  X-ray  department  operations  at  transmitter 
sites,  the  distribution  of  X-ray  department  staff  activities  were 
recorded  using  work-sampling  techniques.  Work-sampling  was  performed 
before  system  installation  and  during  system  operation.  Because  all 
but  one  of  the  transmitter  sites  did  increase  their  staff  to  accommo¬ 
date  system  use  ,  it  was  not  possible  to  measure  directly  the  impact 
of  system  operation  on  the  distribution  of  regular  staff  time.  The 
work-sampling  data  do,  nevertheless,  provide  estimates  of  the  ease 
with  which  the  system  could  be  accommodated  into  normal  department 
operations,  and  served  as  a  replacement  for  alternative  "manual" 
methods  for  obtaining  interpretations. 

Tables  15,  16  and  17  display  the  work  sampling  results.  The 
number  of  minutes  of  staff  time  per  exam  spent  in  various  activities 
varied  among  sites  and  between  the  study  periods.  However,  both 
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before  and  after  system  implementation,  approximately  one-half  of 
X-ray  department  time  was  spent  in  activities  unrelated  to  X-ray 
department  work.  Workload  is  extremely  uneven  in  these  small 
departments,  and  much  of  this  time  was  spent  "on-call"  for  X-ray  duty 
during  non-busy  times.  Performing  and  processing  each  X-ray  and  doing 
the  paperwork  and  filing  associated  with  X-ray  exams  took  between  20 
minutes  and  40  minutes  per  exam;  inputting  teleradiology  images 
required  an  additional  10  minutes  per  exam. 

From  these  data,  it  appears  that  if  a  small  X-ray  department  were 
not  extremely  busy,  it  could  accommodate  teleradiology  system 
operation  into  the  daily  schedule.  Image  inputting  could  be  fit  in  to 
the  day's  routine  —  either  directly  after  examinations  are  performed 
or  during  slow  periods  of  the  day.  However,  if  workload  were  heavy, 
the  10  additional  minutes  per  exam  required  for  teleradiology 
operation  might  present  a  burden. 

2,  Central  Site 

Time  study  data  were  collected  regarding  the  amount  of 
radiologists'  interpretation  time  required  for  viewing  films  and 
viewing  video  images  (Table  18). 

Radiologists  spent  a  mean  of  2.7  minutes  per  X-ray  exam  inter¬ 
preting  films  and  3.40  minutes  per  X-ray  exam  interpreting  video 
images.  There  are  several  possible  explanations  for  this  difference: 

•  video  interpretation  may  actually  require  more  time  than 
film  interpretation; 

•  the  nature  of  the  experimental  teleradiology  system  — 
the  limited  number  of  monitors,  the  number  of  keyboard 
operations  it  required  —  might  have  lengthened  video 
interpretation  time; 

•  the  radiologists'  lack  of  expertise  at  reading  video 
images  might  have  lengthened  video  interpretation  time; 

•  the  two  sets  of  data  may  not  be  equally  accurate:  film 
interpretation  data  were  recorded  by  the  radiologists 
themselves,  while  the  video  data  were  collected  by  the 
teleradiology  system. 
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D.  ACCEPTABILITY  TO  USERS 


During  the  post-implementation  data  collection  period, 
teleradiology  users  were  surveyed  to  determine  their  acceptance  of  the 
system. 

1.  Radiologist  Survey 

The  radiologists  who  interpreted  teleradiology  images  at  the 
central  site  during  the  field  trial  were  asked  to  answer  a  series  of 
open-ended  questions  regarding  their  impressions  of  the  system. 
Twenty-six  (or  87%)  of  the  radiologists  responded.  Approximately 
one-third  (31%)  were  military  physicians;  one-half  described  them¬ 
selves  as  being  in  academic  practice,  and  19%,  in  private  practice. 
Most  of  the  respondents  had  been  involved  in  several  teleradiology 
interpretation  sessions  (56%  in  four  or  more;  30%  in  two  to  three;  and 
14%  in  one) . 

Overall,  the  radiologists  were  impressed  with  the  system. 
Responses  to  specific  questions  are  summarized  below. 

a.  Ease  of  orientation 

Of  the  26  respondents,  only  two  felt  that  orientation  to  the 
system  had  been  difficult  or  awkward.  Eighteen  felt  that  orientation 
had  been  accomplished  very  easily,  and  six,  satisfactorily. 

b.  Ease  of  use 

Overall,  comments  regarding  ease  of  system  use  were  quite 
positive.  Respondents  used  words  like  "excellent,"  "easy,"  "no 
problem,"  "simplistic,"  "straightforward,"  and  "moderate,"  to  describe 
system  operating  procedures.  One  radiologist  felt  that  the  system  was 
awkward  to  use,  and  eight  complained  that  image  accessioning  time  was 
too  long  (though  two  stated  that  this  time  had  been  substantially 
reduced  toward  the  latter  part  of  the  field  trial). 

c.  Technical  quality  of  images  received 

The  general  tone  of  comments  regarding  the  technical  quality  of 
images  was  "acceptable,"  rather  than  "excellent."  Most  felt  quality 
was  "good"  or  "fair";  a  few  said  that  it  was  "moderate"  or  "adequate." 
One  commented  that  transmitted  images  of  soft  tissue  and  chest  exams 
were  not  as  good  as  those  of  bones. 
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d.  Adequacy  of  resolution 

Again,  the  tone  of  responses  was  "acceptable"  rather  than  "good." 
Several  radiologists  commented  that  the  system's  resolution  was 
limited  for  observing  minute  details  and  subtleties  such  as  lung 
changes  or  slight  fracture. 

e.  Accuracy  of  interpretation 

Seven  of  the  respondents  felt  that  their  interpretations  were 
equally  accurate  when  performed  via  video  and  via  film.  Sixteen 
responses  were  positive  but  qualified:  small  fractures  and  fine 

details  were  more  likely  to  be  missed  using  teleradiology,  and 
soft-tissue  images  were  considered  difficult  to  visualize  using  the 
video  mode.  Three  respondents  stated  that  their  video  interpretations 
were  generally  less  accurate  than  their  film  interpretations. 

f.  Confidence  in  interpretation 

Most  respondents  expressed  less  confidence  in  their  video 
interpretations  than  in  those  of  film,  particularly  when  subtle 
abnormalities  were  present  in  the  exam.  They  expressed  a  higher 
degree  of  confidence  for  gross  observations  and  normal  studies,  or 
when  images  were  of  particularly  good  quality.  Two  respondents 
mentioned  that  their  confidence  increased  with  their  increased 
exposure  to  the  video  interpretation  mode. 

g.  Positive  aspects  of  the  system 

Responses  to  this  question  were  quite  diverse.  Respondents 
mentioned  their  ability,  using  the  system,  to  adjust  image  brightness 
and  contrast,  to  avoid  film  handling,  and  to  review  a  large  number  of 
cases  quickly.  They  liked  the  fact  that  technicians  could  transmit 
images  that  were  coned  down  on  the  original  projection. 

h.  Negative  aspects  of  the  system 

Again,  responses  were  diverse.  They  were  generally  focused  on 
the  limited  resolution  of  fine  film  details  (five  comments),  the 
excessive  waiting  time  for  image  display  (eleven  comments),  and  the 
inferior  quality  of  transmitted  images  (four  comments). 

1.  Suggested  improvements 

Not  surprisingly,  the  respondents  suggested  improving  those 
aspects  of  the  system  of  which  they  were  critical:  five  recommended 
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Improvements  in  image  resolution,  seven,  in  the  timeliness  of  image 
display  and  two,  further  technician  transmission  training.  However, 
respondents  also  mentioned  that  two  or  three  more  viewing  monitors 
would  be  useful,  that  a  1024x1024  image  would  be  preferable  to  the 
512x512  used  in  this  system,  and  that  the  range  possible  for  image 
contrast  should  be  expanded.  Also,  human  engineering  refinements  were 
suggested,  involving  redesign  of  the  console  and  simplification  of  the 
keyboard . 

j .  Use  in  remote  clinics 

All  but  two  of  the  respondents  felt  Chat  teleradiology  should  be 
used  to  provide  radiologists'  interpretation  services  to  remote 
clinics  and  hospitals  that  do  not  otherwise  have  access  to 
radiologists'  services.  Only  one  radiologist  felt  that  the  system 
should  definitely  not  be  used  and  one  felt  that  it  should  "possibly" 
be  used.  Five  respondents  qualified  their  affirmative  responses: 
three  recommending  using  the  system  for  wet  readings  only,  one,  for 
emergency  or  urgent  cases  only,  and  one  stressed  that  system  users 
should  be  alerted  to  the  system's  limitations  —  e.g.,  soft-tissue 
resolution,  small  fracture  visualization,  etc. 

k.  Use  in  other  settings 

Four  respondents  suggested  using  the  system  for  teaching,  three, 
for  consultation  within  or  among  hospitals  or  large  outpatient 
clinics,  one,  for  night-call,  and  one,  on  ships  or  in  field  medical 
facilities. 

l.  Additional  comments 

"Additional"  comments  were  generally  enthusiastic:  participants 
felt  that  the  system  was  a  good  start  -  had  great  potential, 
especially  in  military  settings.  Two  radiologists  commented  that  this 
system  would  be  better  for  serving  an  uncomplicated  outpatient 
population  like  that  involved  in  the  field  trial,  rather  than  for 
providing  basic  interpretation  services  to  facilities  that  would 
require  radiologists  to  provide  other  services,  i.e,  for  fluoroscopic 
exams  or  other  special  procedures. 
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2.  Provider  Survey 

A  written  questionnaire  regarding  post-implementation  X-ray 
services  was  completed  by  28  (90%)  of  the  health  care  providers  at  the 
transmitter  sites.  Seventy-five  percent  of  those  responding  were 
staff  physicians,  21%  were  physicians'  assistants  and  4%  were  nurse 
practitioners.  Respondents  were  generally  pleased  with  the  X-ray 
services  at  their  facilities,  and  impressed  with  the  teleradiology 
system's  potential  utility. 

a.  Utilization  of  radiologists'  services 

Ninety-three  percent  of  respondents  stated  that  they  always  read 
the  radiologist's  X-ray  interpretation  report  on  exams  that  they  order 
(while  7%  read  this  report  "most  of  the  time").  Most  said  that  they 
used  their  own  film  interpretation,  as  well  as  the  radiologist's,  in 
making  diagnostic  or  treatment  decisions  (41%  depend  on  their  own;  33% 
use  the  radiologist's  combined  with  their  own;  and  26%  depend  on  the 
radiologist's  interpretation).  All  respondents  stated  that  they  did, 
at  times,  consult  with  radiologists  (Seventy-eight  percent  do  this  to 
discuss  X-ray  films,  59%  to  clarify  reported  findings  and  to  determine 
the  need  for  further  examinations,  37%  to  consult  in  advance  regarding 
the  need  for  X-ray  and  33%  to  ask  for  information  not  found  in  the 
X-ray  report).  Thirty-seven  percent  felt  that  the  non-availability  of 
radiologists  at  their  facility  represented  a  major  drawback  to  X-ray 
service,  while  44%  felt  that  delayed  reporting  was  the  X-ray 
department's  primary  shortcoming. 

b.  Opinions  of  X-Ray  services 

Providers  were  asked  to  note  how  often  they  experienced  delays  in 
receipt  of  X-ray  reports  or  in  finding  X-ray  films,  and  to  indicate 
how  often  these  incidents  delayed  patient  management.  Survey 
responses  are  summarized  in  Table  19,  While  most  providers  (70%) 


During  the  baseline  period,  Arthur  D.  Little  was  not  responsible 
for  surveys  of  providers,  but  such  surveys  were  performed  and 
analyzed  by  the  Bureau  of  Radiological  Health  and  results  are 
available  in  the  BRH's  report  of  findings. 
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indicated  that  they  had  experienced  at  least  one  reporting  "delay" 
during  the  past  month,  these  delays  usually  did  not  result  in  patient 
management  being  postponed.  A  much  smaller  proportion  of  providers 
(33%)  had  experienced  delays  in  finding  X-ray  films,  but,  once  again, 
seldom  did  delays  affect  patient  care.  The  majority  of  providers  felt 
that  delays  occurred  with  similar  frequency  during  the  field  trial  and 
prior  to  system  installation;  a  number  felt  that  there  were  fewer 
delays  since  teleradiology  had  been  installed. 

Fourteen  percent  of  providers  indicated  that  they  had  recently 
(i.e.  during  the  field  trial)  had  to  re-order  an  X-ray  exam  because  of 
delays  in  reporting,  while  7%  indicated  that  a  re-examination  had  been 
required  because  of  difficulty  in  finding  films. 

Providers  were  asked  to  rate  various  aspects  of  the  X-ray 
services  provided  at  their  facilities.  These  ratings  are  summarized 
in  Table  20.  Overall,  answers  were  quite  positive.  Providers  were 
most  concerned  with  the  timeliness  of  reporting  (21%  indicated  that 
timeliness  was  "poor"),  but  the  majority  felt  that  the  timeliness, 
availability,  accuracy,  comprehensiveness  and  readability  of  reports 
were  either  "good"  or  "excellent."  They  felt  that  X-ray  department 
staff  were  both  able  and  cooperative,  but  noted  some  problems  with 
patient  scheduling  and  with  the  availability  of  X-ray  films. 
c.  Teleradiology 

Providers  were  asked  several  questions  concerning  their 
familiarity  with  and  opinions  of  the  teleradiology  system.  All 
respondents  stated  that  they  were  aware  of  the  system's  existence  and 
most  (54%)  felt  that  it  had  improved  X-ray  services;  however,  10%  felt 
that  the  system  had  aggravated  or  created  problems  in  the  department. 

Although  several  providers  commented  that  the  teleradiology 
system  had  great  potential,  few  felt  that  it  had  had  much  impact  on 
X-ray  services  or  on  patient  care  during  the  field  trial.  Two 
commented  that  the  present  system  had  had  too  many  technical  problems 
to  be  really  useable;  three,  that  because  teleradiology  interpreta¬ 
tions  had  not  generally  been  received  prior  to  film  interpretations, 
the  system  had  had  little  impact. 
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Concerning  future  use  of  teleradiology,  two  providers  commented 
that  rapid  turnaround  would  be  of  little  value  in  primary  care  clinics 
such  as  theirs,  because  providers  rely  on  their  own  interpretations  of 
simple  cases  and  refer  complex  cases  elsewhere  for  both  diagnosis  and 
treatment.  Three  commented  that  a  1-hour  report  turnaround  would  be 
very  useful,  one  praising  the  PRIORITY  mode,  which  had  been  added  to 
the  system  during  the  latter  half  of  the  trial.  One  felt  that  prompt 
radiologists'  interpretations  would  be  most  useful  to  emergency  room 
providers  during  evenings  and  weekends,  when  other  providers  were 
unavailable  to  assist  in  film  interpretation  and  disposition 
decisions.  And  one  recommended  that  a  telephone  be  available  for 
communication  with  interpreting  radiologists. 

3,  User  Interviews 
a.  Technicians 

Each  of  the  transmitter  site  system  operators  was  interviewed 
during  the  post-implementation  data  collection  period  to  determine 
their  opinions  of  the  system.  A  total  of  eight  individuals  were 
interviewed:  seven  technicians  who  operated  the  transmitter  systems 

at  the  sites  and  one  specially  hired  transmitter  system  operator.  The 
interviews  focused  on  how  well  the  system  had  functioned,  ease  of 
system  use,  the  adequacy  of  system  turnaround  time,  exams  for  which 
the  system  was  most  useful,  the  impact  of  the  system  on  patient  care, 
suggestions  for  ways  of  improving  the  system,  and  opinions  on  whether 
and  where  the  system  should  be  permanently  installed.  Responses  are 
summarized  below. 
i.  System  function 

Users  reported  that  the  system  had  functioned  reasonably  well 
during  the  latter  half  of  the  field  trial.  Early  on,  both  the  image 
processing  system  and  the  word  processor  had  frequently  malfunctioned. 
The  fewest  problems  were  experienced  at  Bolling,  where  the  system  was 
not  installed  until  quite  late  in  the  field  trial,  and  the  most  were 
found  at  CVCHC  —  where  the  telephone  lines  were  poor  —  and  Patuxent 
—  which  had  the  largest  workload. 
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il.  Ease  of  system  use 

All  of  the  operators  reported  that  the  system  was  very  easy  for 
them  to  use,  although  several  commented  that  the  inputting  process  was 
too  time  consuming.  Some  also  complained  about  the  number  of  manual 
and  physically  awkward  steps  involved  in  inputting  films  and  the  time 
required  for  logging  activities,  recording  patient  information  and 
operating  the  printer.  Only  one  of  the  system  operators  felt  that  the 
software's  many  prompts  and  safeguards  were  superfluous;  the  others 
fount  these  reminders  helpful. 

iii.  Turnaround  time 

Technicians  generally  felt  that  the  system’s  several-day 
turnaround  time  for  interpretation  reports  was  adequate  for  their 
facilities  because  most  of  the  cases  that  were  input  were  not 
associated  with  emergencies.  Three  commented  that  a  1-hour  turnaround 
time  would  greatly  enhance  the  system's  utility;  they  felt  that  the 
PRIORITY  mode  had  not  been  available  for  long  enough  to  be  given  an 
adequate  trial. 

iv.  Examinations  for  which  the  system  was  most  useful 

Three  technicians  reported  that  the  system  was  useful  for  all 
exams  equally,  while  others  specified  those  for  which  teleradiology 
was  most  or  least  appropriate.  Several  technicians  felt  that  the 
system  would  be  best  for  emergency  exams,  but  one  technician  commented 
that,  because  emergency  cases  were  usually  sent  elsewhere  for 
treatment,  timely  reporting  was  not  a  high  priority.  Technically,  the 
technicians  felt  that  the  system  transmitted  extremity  films  and 
fractures  best,  and  chest,  spine,  hand  and  abdomen  exams  less  clearly. 

v.  Impact  on  patient  care 

Technicians  did  not  generally  feel  that  the  system  had  affected 
patient  care.  Two  reported  that  for  the  mix  of  patients  examined  at 
their  clinics,  the  system  could  have  little  impact,  even  if  turnaround 
time  were  reduced.  Others  felt  that  the  system's  impact  was  limited 
by  referring  providers'  lack  of  confidence  in  the  system. 
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vl.  Suggested  Improvements 

The  technicians  had  many  suggestions  for  system  improvement,  most 
of  which  were  designed  to  shorten  the  inputting  process,  and  some  to 
increase  the  amount  of  information  that  was  transmitted.  Major 
recommendations  were: 

•  quadrant  inputting  should  be  automatic 

•  image  focusing  should  be  automatic 

•  software  redundancy  should  be  reduced 

•  time  lag  between  images  should  be  reduced 

•  control  of  all  inputting  functions  should  be  possible 
from  the  keyboard 

•  patient  and  case-specific  information  should  be  recallable 
and  revisable  during  or  after  image  capture 

•  more  projection  codes  should  be  available  and  free  form 
descriptions  of  projections  should  be  possible 

•  lightbox  lighting  should  be  more  consistent 

•  the  lightbox  should  be  larger 

•  the  images  should  be  markable  with  an  R  or  an  L  and  ruled 

•  the  system's  sensitivity  to  external  problems,  such  as 
rain  or  telephone  line  inadequacies  should  be  reduced 

•  staffing  should  be  increased  to  accommodate  system  use 
vii.  Permanent  installation 

All  of  the  system  operators  felt  that  they  would  like  to  see  a 
refined  teleradiology  system  permanently  installed  in  their  facili¬ 
ties.  Several  commented  that  it  would  also  be  useful  on  ships,  in 
facilities  that  process  large  numbers  of  physical  examinations,  and  in 
areas  that  do  not  otherwise  have  radiologists  services  available. 
b.  Project  Officers 

Each  of  the  project  officers  at  the  transmitter  sites  was 
interviewed  regarding  his  opinions  of  the  system.  Three  were 
physicians  and  one  was  an  administrator.  On  most  points,  the  project 
officers’  opinions  correlated  with  those  of  the  technicians.  However, 
they  had  more  comments  regarding  the  system's  potential  and  the 
clinical  utility  of  the  system.  Major  impressions  were: 
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•  The  system  is  more  likely  to  increase  the  clinical 
confidence  of  providers  than  to  alter  actual  patient 
treatment. 

•  The  system  is  of  more  value  where  referring  providers  are 
less  capable  of  interpreting  films  themselves. 

•  The  system  would  be  particularly  useful  if  it  allowed 
24-hour  access  to  radiologist  consultation. 

•  The  system  can  serve  to  extend  the  arm  of  the  remote 
primary  care  provider,  allowing  him  to  function  as  he 
would  in  the  outpatient  department  of  a  hospital. 

Several  project  officers  expressed  disappointment  with  the  system, 
having  expected  a  high  degree  of  functional  reliability  and  a  very 
short  report  turn  -ound  time. 

E.  ESTIMATED  INCREMENTAL  COSTS  OF  USING  TELERADIOLOGY 

In  order  to  determine  the  cost  of  using  the  teleradiology  system 
in  facilities  typified  by  the  transmitter  sites  and  the  central  site 
involved  in  the  field  trial,  data  were  collected  regarding  total  X-ray 
department  operating  costs,  and  the  costs  of  teleradiology  system. 
Also,  the  costs  of  obtaining  radiologists'  interpretations  by  the 
standard  "manual"  methods  used  at  the  transmitter  sites  were  estimated 
from  observation  of  their  operations.  Estimated  annual  costs  of 
operating  the  X-ray  services  at  the  transmitter  sites  —  less  the  cost 
of  obtaining  radiologists*  interpretations  —  are  presented  in  Table 
21.  These  figures  were  combined  with  X-ray  workload  to  calculate  cost 
per  examination  at  each  facility,  as  shown  in  Table  22.  The  per 
examination  costs  can  be  used  as  base  figures  against  which  to  compare 
the  costs  of  obtaining  radiologists'  interpretation  services  using 
different  methods. 

At  each  of  the  four  transmitter  sites,  it  would  be  possible  to 
use  any  of  three  standard  methods  for  obtaining  radiologists'  interpre¬ 
tation  services:  to  use  a  courier  service,  to  contract  for  a  visiting 
radiologist  on  a  part-time  basis  or  to  use  teleradiology.  In  these 
clinics,  the  equipment  and  staff  necessary  to  perform  and  process 
X-ray  exams  are  essentially  the  same  regardless  of  which  method  is 
used.  Hence,  in  the  analysis  of  teleradiology's  costs,  attention  was 
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focused  only  on  the  incremental  or  marginal  costs  associated  with  the 
three  alternatives;  only  the  costs  of  obtaining  the  interpretation 
were  included;  those  of  performing  the  interpretation  or  other  X-ray 
processing  activities  were  not. 

Table  23  shows  estimates  of  the  incremental  costs  of  obtaining 

interpretation  services  at  facilities  similar  to  the  study  sites  using 

13 

each  of  the  three  methods  listed  above.  The  cost  of  using  the 
teleradiology  system  was  found  to  be  approximately  $7  per  X-ray  exam, 
compared  with  an  estimated  $2.50  per  exam  when  a  part-time  visiting 
radiologist  is  employed  and  $0.50  when  a  courier  system  is  used. 

The  figures  presented  in  Table  23  were  based  on  the  characteris¬ 
tics  of  the  field  trial  transmitter  sites,  on  the  central  site,  and  of 
the  teleradiology  system.  These  figures  would  vary  in  different 
settings.  For  example,  operating  a  courier  or  visiting  radiologist 
system  would  be  much  more  expensive  if  the  clinics  were  very  remote. 
And  the  cost  of  using  a  teleradiology  system  would  be  reduced  if  the 
capital  cost  of  the  system  was  lower  or  if  a  larger  volume  of  cases 
were  input  into  the  system;  and  teleradiology  costs  would  increase  if 
a  satellite  were  used  for  image  transmission  or  if  transmitter  system 
operators  went  through  a  formal  training  program. 


Tables  C-10  and  C-ll  in  the  Appendix  present  cost  figures  for  the 
teleradiology  system  components. 
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Estimated  workload  of  a  typical  small  X-ray  department 


V.  DISCUSSION'  AND  CONCLUSIONS 


The  nature  of  the  field  trial  did  not  allow  direct  measurement  of 
the  patient  care  impacts,  staffing  impacts,  or  cost  impacts  of 
teleradiology  use.  However,  the  results  of  the  trial  can  be  used  to 
reach  some  conclusions  about  the  utility  of  teleradiology  in  routine 
medical  practice. 

First,  the  teleradiology  system  appears  to  be  acceptable  to 
users.  A  few  human  engineering  refinements,  and  increased  exposure  to 
the  system  would  probably  make  it  totally  acceptable  operationally. 

Second,  operating  the  teleradiology  system  requires  about  10 
minutes  per  exam.  In  a  small  X-ray  department  whwere  workload  is 
moderate,  technicians  would  be  likely  to  have  enough  time  to  perform 
their  regular  duties  and  to  operate  the  machine  as  well.  If  workload 
were  heavy,  however,  the  10  additional  minutes  might  present  a  burden 
to  technicians. 

It  appears  that  the  interpretation  of  teleradiology  images 
requires  slightly  more  time  than  interpreting  exams  in  film  form. 
This  incremental  time  is  small,  however,  and  it  is  probable  that  if  a 
large  X-ray  department  could  accommodate  the  film  interpretation  of  a 
given  number  of  X-ray  exams,  it  could  similarly  accommodate  the 
interpretation  of  teleradiology  images  of  these  exams. 

Third,  if  operating  procedures  were  redesigned  to  maximize  the 
benefits  of  the  system,  teleradiology  report  turnaround  time  could  be 
quite  short:  24-hours  for  routine  exams  and  1  hour  for  STAT  cases. 
Such  procedures  would  allow  small  or  remote  clinics  to  have  access  to 
radiologists'  services  similar  to  that  which  exists  in  larger  or  less 
remote  facilities  (such  as  hospital  outpatient  departments).  It  is 
important  to  recognize  that  a  teleradiology  system  can  only  provide 
rapid  report  turnaround  if  it  is  operated  in  a  very  routine  and  timely 
fashion . 

F  urth,  the  major  patient-care  benefits  that  could  result  from 
using  teleradiology  is  a  reduction  in  turnaround  time  for  the 
radiologist's  interpretation  report.  The  patient-care  value  of  the 
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system  depends  upon  how  important  the  promptness  of  the  turnaround  is 
to  patient  treatment  and  disposition  decisions  in  a  given  setting. 
For  any  setting,  the  value  of  prompt  interpretation  depends  on: 

•  whether  many  X-ray  are  performed  for  use  in  immediate 
treatment  decisions  or  are  performed  routinely  or 
"for  the  record"; 

•  the  relative  interpretation  skill  of  the  health  care 
providers  who  would  otherwise  be  interpreting  the  exams. 

The  less  confident  and  the  less  skilled  the  providers, 
the  more  valuable  is  the  prompt  expert's  interpretation. 

•  the  treatment  capabilities  of  the  facility  where  the  X-ray 
is  performed.  If  patients  who  are  severely  ill  can  not  be 
be  treated  adequately  ar  a  given  transmitter  site  prompt 
interpretation  receipt  is  of  marginal  value  since  such 
patients  are  immediately  sent  elsewhere  regardless  of 
X-ray  findings.  On  the  other  hand,  if  X-ray  results  may 
help  providers  to  decide  whether  or  not  to  send  patients 
elsewhere,  prompt  interpretation  may  be  very  important. 

•  the  alternatives  available:  in  extremely  remote  locations, 
teleradiology  may  be  the  only  method  for  obtaining 
radiologists'  services,  or  obtaining  them  in  a  reasonable 
length  of  time,  while  in  less  Isolated  settings,  other 
alternatives  may  be  available. 

Fifth,  the  costs  of  using  the  teleradiology  system  appear  to  be 
higher  than  the  costs  of  the  manual  alternatives  for  facilities 
typified  by  the  field  trial  transmitter  sites.  However,  the  relative 
costs  of  the  three  alternative  methods  for  obtaining  interpretations 
would  vary  in  different  settings. 
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APPENDIX  A 


DATA  COLLECTION  INSTRUMENTS 


A-l 


Arthur  EX  Little,  Inc. 


Data  Collection  Instrument  X-l 


Time  History:  X-Ray  Request /Report  Cycle 


A- 3 


/L.  Arthur  D.  Little,  Inc. 


TIME  HISTORY: 

X-RAY  REQUESTS/REPORT  CYCLE 


Arthur  D.  Little,  Inc,,  hat  been  commissioned  by  the  TRIMIS  Program  Office  to  evaluate  impacts  of  a  TELERADIOLOGY 
system  on  the  rediology  service  provided  in  several  clinics  in  the  Washington,  D.C.  area.  This  system  it  scheduled  for  in¬ 
stallation  in  the  late  Fall  of  1981. 

As  part  of  the  evaluation,  we  are  interested  in  describing  the  present  system  for  requesting  end  interpreting  X-rays.  We  are 
asking  staff  to  complete  the  various  sections  of  the  attached  form,  which  document  the  five  major  stages  of  the  X-ray 
request/report  cycle: 

to  the  ATTENDING  PHYSICIAN: 

SECTION  I  is  to  be  completed  by  the  ATTENDING  PHYSICIAN  when  ordering  an  X-ray  examination, 
to  the  X-RAY  TECHNICIAN: 

SECTION  II  it  to  be  completed  by  the  X-RAY  TECHNICIAN  when  performing  the  X-ray  examination, 
to  the  ATTENDING  PHYSICIAN: 

SECTION  III  is  to  be  completed  by  the  ATTENDING  PHYSICIAN  if  and  when  he  looks  at  the  films  before  a  radiol¬ 
ogist  interprets  them. 

to  the  RADIOLOGIST: 

SECTION  IV  is  to  be  completed  by  the  RADIOLOGIST  performing  the  X-ray  interpretation. 

to  the  RECEPTIONIST  or  TECHNICIAN  at  the  originating  clinic: 

SECTION  V  it  to  be  completed  when  the  interpretation  report  is  received  at  the  originating  clinic  (if  the  X-ray  has 
been  sent  elsewhere  for  interpretation). 

to  the  ATTENDING  PHYSICIAN: 

SECTION  VI  is  to  be  completed  when  the  interpretation  is  reviewed  by  the  ATTENDING  PHYSICIAN. 

(We  understand  that  some  questions  may  not  apply  to  every  clinic.) 

Information  will  also  be  collected  after  the  TELERADIOLOGY  system  is  operational.  The  data  will  be  used  to  identify 
changes  in  the  use  of  X-ray  examinations  in  patient  care. 


THANK  YOU. 


(1-4) 


SECTION  I:  To  be  filled  out  by  the  ATTENDING  PHYSICIAN  when  ordering  X-rays. 

1.  Whet  is  the  reason  for  this  X-ray  examination 

Routine  physical  examination  (1) . . -  (61 

For  the  record :  diagnostic,  but  not  expected  to  affect  patient's  immediate  disposition.  Please  note  (2)  -  .... 
presenting  symptoms  or  provisional  diagnosis: 


Diagnostic:  expected  to  influence  petient's  immediate  disposition.  Please  note  presenting  symptoms  (3) 
or  provisions!  diagnosis: 


Follow-up. 


(4) 


2 


Given  that  the  option  existed  of  having  these  films  interpreted  by  a  radiologist  within  one  hour  of  exam  performance,  would 
you  feel  that  such  a  prompt  reading  would  be: 

Not  significant.  Would  not  affect  patient's  treatment  or  immediate  disposition.  (1) -  (6) 

Somewhat  significant.  Unlikely  to  affect  patient's  treatment  or  immediate  disposition,  but  likely  to  12) - 

increase  your  diagnostic  confidence. 


Significant.  Likely  to  affect  your  opinions  and/or  decisions  regarding  patiant  care  and/or  disposition.  (3) 
Very  significant.  Essential  to  your  decisions  regarding  patient  care  and/or  disposition.  (4) 


3. 

TIME  of  request  for  X-ray  examination: 
hour  minute 

month 

day 

year 

/  /  /  /  / 

/  ✓  / 

/  /  / 

L  /  J 

(7-8)  (9-10) 

(11-12) 

(13-14) 

(16-16) 

(24-hour  clock,  e.g.,  1645) 

(Date,  e.g.,  09/23/81) 

4.  Immediate  disposition  of  patient: 


(17) 


M fait  in  clinic  for  attending  physician  to  interpret  X-rays.  —  - .  . .  -  -  —  (1) 

Wait  in  clinic  for  radiologist  to  interpret  X-rays.  — — — —  •  —  ■  -  -  -  -  (2) 

Wait  in  clinic  for  other  reasons.  -  (3) 

Discharged.  Return  to  normal  activities  -  14) 

Discharged.  Stay  at  home. -  IS) 

Referred  to  another  facility. - —  (6) 

Admitted  for  treatment  or  observation. -  (7) 

Return  to  ward - —  (8) 


THANK  YOU. 


ATTENDING  PHYSICIAN  or  P.A.  -  SECTION  I 


SECTION  II:  To  be  filled  out  by  the  X-RAY  TECHNICIAN. 


& 

FACILITY 

Bolling  AFB  Clinic 

ID  - 

-  (18) 

Central  Virginia  Community  HC 

(2)  - 

— 

Ft  Detrick  Hospital 

13)  - 

— 

Patuxent  NATC 

(4)  - 

— 

6. 

Patient's  Identification 

(20-30) 

Numl-r*  ////-//  /-/  /  J 

:  /  /-/_  /_y 

7. 

A*  /  L./ 

(31-33) 

& 

Sex: 

Female 

(1)  - 

-  (34) 

Male 

(2)  - 

— 

9. 

Status: 

Outpatient 

(1)  - 

-  (36) 

Inpatient 

(2)  - 

— 

Emergency 

(3)  - 

— 

10. 

Exam  typa(s)  requested  (e.g.,  skull,  abdomen,  chest) 

1. 

_  (36) 

2. 

(37) 

3. 

(38) 

4. 

(39) 

5. 

(40) 

11. 

Is  this  a  NEW  patient? 

Yes 

(1)  - 

-  (41) 

No 

(2)  - 

— 

12.  If  this  n  NOT  a  now  patient,  how  many  pertinent  old  films  are  available  li.e..  relatively  recent  film)  of  the  tame 
area  of  the  body)? 


none 

(0) 

three 

(3) 

five  to  ten 

(6)  - 

-  (42) 

one 

in 

four 

(4) 

eleven  to  fifteen 

(7)  - 

— 

s-wn 

(2) 

five 

(5) 

sixteen  to  twenty 

(8)  - 

— 

more  than  twenty 

(9) - 

- 

‘Patient  number  will  be  uied  only  to  correlate  responses  and  will  then  be  deleted  in  future  analysis 


THANK  YOU. 


X-RAY  TECHNICIAN  -  SECTION  II 


SECTION  III:  To  be  filled  out  by  the  ATTENDING  PHYSICIAN  if  and  when  looking  at  the  film t  before  a  radiologist 
interprets  them. 

1 3.  How  has  your  viewing  of  thasa  films  affectad  your  handling  of  this  case? 

\o  effect  or  inconclusive.  _ _ _  (1)  _ _  (43) 

Some  effect:  increased  confidence  in  clinical  impression.  .  (21 _ _ 

Some  effect:  altered  clinical  impression,  but  did  not  alter  patient  treetment  and/or  disposition _ (3)  _ 

Major  affect:  altered  patient  treatment  and/or  disposition. _ _ _  (4)  _ 


14.  Disposition  of  patient: 

Disposition  already  made.  __ _ _ _ _ _ (1)  _ _  (44) 

Wait  in  c/inic  for  radiologist  to  interpret  X-rays. _  (2) _ 

Wait  in  clinic  for  other  reasons.  _ _  (3)  _ _ 

Discharged.  Return  to  normal  activities  _ _ _  (4) _ 

Discharged.  Stay  at  home.  — - — _ _ _ _ _  (5)  _ _ 

Referred  to  another  facility.  _ _ _ _  (6)  _ _ 

Admitted  for  treatment  or  observation. . . .  (7)  _ 

Return  to  ward. _ _ _ _ _  (8)  _ 


THANK  YOU. 


ATTENDING  PHYSICIAN  or  P.A.  -  SECTION  III 


SECTION  IV:  To  be  filled  out  by  the  RADIOLOGIST  interpreting  the  X-ray  films. 

15.  LOCATION  OF  RADIOLOGIST: 

Bethesda  NMC  (1) 

Ft.  Detrick  Hospital  (21 
Malcolm  Grow  Hospital  (3) 
Patuxent  NATC  (4) 

University  ol  Virginia  MC  (5) 
Other  (6) 

16.  How  many  OLD  films  did  you  use  for  comparison  in  this  interpretation? 

none  (0) 

one  ( 1 ) 

two  (21 

three  (3) 

four  (4) 

five  (5) 

five  to  ten  (6) 

eleven  to  fifteen  (71 
sixteen  to  twenty  (8) 
more  than  twenty  (91 


THANK  YOU. 


RADIOLOGIST  -  SECTION  IV 


SECTION  V:  To  be  filled  out  by  the  RECEPTIONIST  or  TECHNICIAN  at  the  X-ray's  ORIGINATING  CLINIC  at  the 
time  of  receipt  of  the  X-ray  report  (if  films  were  sent  outside  for  interpretation). 

year 

LJ-J 

(55-56) 


THANK  YOU. 


RECEPTIONIST  or  TECHNICIAN  -  SECTION  V 


TIME  interpretation  report  received  at  originating  clime 

hour  minute  month  day 

/  /  /  /  /  /  /  /  LJ-J 

(47—46)  (49—501  (51-52)  (53-54) 


(47-48)  (49-50) 
(24-hour  clock,  e.o 


(Date.  e.o..  09/23/81) 


f 


SECTION  VI:  To  be  filled  out  by  the  ATTENDING  PHYSICIAN  at  the  time  of  X-ray  interpretation  report  receipt. 

18.  TIME  interpretation  report  reviewed  by  attending  phytician: 

hour  minute  month  day  year 


157-68)  (59-60) 

(24-hour  clock,  e.g.,  1645) 


month  day 

f  /.  /  LJ.J 

(61  -62)  (63-64 

(Date,  e.g.,  09/23/81) 


Did  the  radiologitt  communicate  hi>  interpretation  to  you  verbally? 


(65-66) 


If  yes,  at  what  time? 
hour  minute 


(68-69)  (70-71) 

( 24-hour  clock,  e.g.,  1645) 


month  day 

L-LJ  Lmj 

(72-73)  (74-75) 

(Date,  e.g.,  09/23/81) 


(76-77) 


If  yes,  by  what  method? 


Oral /personal 
Telephone 


How  did  the  radiologist's  report  (written  or  oral)  affect  your  handling  of  this  cate; 

No  effect.  Merely  duplicated  your  own  findings.  _ _  (1)  _ 

Some  effect.  Increased  your  confidence  in  your  interpretation _  (2)  _ 

Some  effect.  Altered  your  opinion  but  did  not  alter  course  of  patient  treatment  and/or  disposition _ (3)  _ 

Major  effect.  Altered  course  of  patient  treatment  and/or  disposition.  _  (4)  _ 

Major  effect.  Caused  recall  of  patient  to  reaffirm  clinical  impression. _ _  (5)  _ 


Disposition  of  patient: 

Disposition  already  made.  _ _  ( i )  _ 

Wait  at  dime  for  other  reasons. _ _ (2)  _ 

Discharged.  Return  to  normal  activities  _ _ _ _  (3)  _ 

Discharged.  Stay  at  home.  _ _ _ _  (4)  _ 

Referred  to  another  facility.  _ _  (5)  _ 

Admitted  for  treatment  or  observation. _ _ (6)  _ 

Return  to  Heard. _  .  _ _  (7)  _ 


If  you  had  received  the  radiologist's  report  on  the  day  of  the  X-ray  exam,  do  you  feel  that  patient  treatment  or  disposition 
would  have  been  altered  (presuming  that  you  did  not,  in  fact,  receive  the  report  this  promptly)? 

Yes  (1)  - 

No  (2)  - 

If  your  answer  to  question  24  was  No,  please  proceed  to  question  26. 


26.  If  you  had  received  the  radiologist's  report  within  one  hour  of  exam  performance,  do  you  feel  that  patient  treatment  or 
disposition  would  have  been  altered  (presuming  that  you  did  not,  in  fact,  receive  the  report  this  promptly)? 


If  no.  please  disregard  question  27. 


27.  If  yet,  how? 


Yes  (1) 
No  (2) 


THANK  YOU. 


ATTENDING  PHYSICIAN  or  P.A.  -  SECTION  VI 


Data  Collection  Instrument  Y-l 
Time  History:  X-Ray  Request/Report  Cycle 

A:  Film  X-Ray  Request/Interpretation  Report 
B:  Video  Image:  Entry/Interpretation 
C:  Video  Image:  Interpretation/Transcription 
D:  Video  Image:  Interpretation  Report 


A- 1  ^ 


t  \t<h 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS- I963-A 


TIME  HISTORY: 


X-RAY  REQUEST/REPORT  CYCLE 


Arthur  D.  Little,  Inc.,  has  been  commissioned  by  the  TRIMIS  Program  Office  to  evaluate 
impacts  of  the  TELERADIOLOGY  system  on  the  X-ray  service  provided  in  several  clinics  in 
the  Washington,  D.C.  area. 

As  part  of  the  evaluation,  we  are  asking  staff  to  complete  the  various  sections  of  the 
attached  form,  as  it  follows  the  X-ray  request  from  initiation  to  final  report  receipt. 


THANK  YOU. 


1  ‘j*  ,  #s~  r-  '  -**,  jw*w 

’  -’.S-  .^v."  ’  .  ‘  V  .^/ *•>  -  v'::  ;  '  ,<«V' ^ :-^v:- ■,' 


1*.  TMndWon  i 


I 

I 

IP.  Tatd 
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ft  Wei 

Jt»  mmt 

£umber< 

^umbtrt 


SECTION  IV:  to  bo  AIM  out  by  TECHNICIAN  or  TELERAOIOLOGY  SYSTEM  OPERATOR 

at  tim«  of  TELERAOIOLOGY  ENTRY 


hllf— iRd  In  ImffMBO  Utiii  you  haya  imwi*  ngtiw  tw*  H»to  i»—  Pla^iuilft 

roUtae  (OLD  pnd  NEW)  end  (he  number  of  image*  (of  Mam  OLD  and  NEW)-- 


Lumber*  OLD  FILMS 
(11-1*) 

not  NEW  FILMS 
(IP-14) 


LU 


number  of  IMAGES  of  OLD  FILMS  I  I  I 
(1P-1C)  I — I — * 

number  of  IMAGES  of  NEW  FILMS  .  .  . 

(iMo  Li I 


(1)  —  Yee 

(2)  —  No 


Ihan 


IK  YOU. 


IV.  TECHNICIAN  OR  TELERAOIOLOGY  SYSTEM  OPERATOR:  At  time  of  TELERADIOLOGY  ENTRY 


SECTION  V:  to  bo  tilled  out  by  RECEPTIONIST  at  time  of  FILM  RECEIPT 


I 

L 

I 

1 


TKJE  turn*  received  at  Mad  leal  Center  for  Interpretation: 


TIME 

DATE 

month 

(My 

l  1  1  1  1 

LLJ 

LU 

(24-hour  clock. 

(•*. 

0  5) 

e.g.,  IMS) 

o  e 

1 

1 

1 

.THANK  YOU. 


■  V.  RECEPTIONIST  AT  MEDICAL  CENTER:  At  time  of  FILM  RECEIPT  (MGMC  and  NNMC  only) 


2  3  $  3  S  3 


SECTION  VI:  le  to  fltod  out  by  RAMOLOQ0T  at  3m  of  MJI  INTERPRETATION 


(3)  _ tm  (6)  —  six  to  ten 

(4)  _ tour  (7) _ eleven  to  fifteen 

(5)  _ Ova  (8) _ sixteen  to  twenty 

(9) _ more  than  twenty 


_ FLDsMek 

_ Matootn  Qrow  MC 

_ PatuMsnt  NATO 

w  «*--»-■-  «v» 
Islv^pNilSI 

_ Otoer 


VI.  RAOIOLOQIST:  At  ttm*  0<  FILM  INTERPRETATION 


SECTION  VII:  to  be  filled  out  by  TRANSCRIPTIONIST  at  time  of  REPORT  TRANSCRIPTION 

(if  report  Is  transcribed) 


TIME  typing  of  X-ray  roport  bagun: 


TIME 

DATE 

month 

day 

LI  1  1  1 

LU 

LU 

(24-hour  dock, 

(o.g., 

0  5) 

o.g.,1645) 

0  6 

1  Li-U 

LU 

LU 

fHANK  YOU. 


II.  TRANSCRIPTIONIST:  At  time  of  REPORT  TRANSCRIPTION  (If  report  Is  transcribed) 


SECTION  VM:  to  to  filled  out  by  RADIOLOGIST  at  Unto  of  REPORT  SIGNING 

(if  report  is  signed) 


1  LUJ 


(24-hour  dock, 
1648) 


DATE 

month  day 

LU  LU 

(•». 

o  e  0  5) 


VM.  RADIOLOGIST:  At  time  of  REPORT  SIGNING  (If  report  Is  signed) 


♦  '"A 


f 

V 

i 


3£K  ^ ** 


-  ‘V  \ 


:  '•  ' 


ft)  —  Raakaot  Manly  «fMM  year  awn 

(3|  ammrntt  mum  yaw  apwan  hu>  *>  na  a> 
(O — 

»  — 


r 


cm 

ro¬ 


ta  you 


I _ No  (V  NO, 


atop  to  quiNton  #  33) 


_«-  RVW.by«M 

cm 


r  <’>— 

(ft  . —  Tataphona  • 
a  >  Yaa,  at  artiat  TIME7 

Lllu 


(M> 

m 


<•» 

r 


to  — 

l  m~ 1 

"•  (7)  _  I 


m.W  -?$ 


DATE 


LI -I  U  LULU 


( nm 


(1)  —  Sky  to  cMc  tar  Curthor  toata,  traalmant  or  otoaarvaMon. 

(3)  —  Ltmm  ctotto.  RMum  to  nomwl  acdyMta*. 

(S)  —  kM  oktte.  Oo  not  ralum  to  normal  adMtlaa. 

*"  (4|  —  Ltmm  c*nta.  Go  to  anotaar  madtaal  taeMly. 

«Aa  aa^BkAuJ  4gk  —  —  nr,  ^  —  n  — *t  -  - 

i  ■  u  ™  w  M  uNRnlfK  Of  OOMcVtVOn. 

nmrn  to  ward. 

(7)  —  Mo  taNftarlnaaucaontarBnacaaaafy,aapB»ant  did  not  ataytncUntc  for  radtotoglirg  Interpretation  to  bo 


T  W  —  Okar.  apaeay: 

Ja 


rtf  kku  tfutf  aatotffltarf  a  agtotfMAfwaitoA*  a  MMkl  Anna  Ta^MtfnbiMu  nr  itoa  Bkto  -  **  -- 

■  mm  ■  mvBftf  ■  i^tfn  \wf  iiwwiw^y  w  Bf  mi  niw|NVUDOn) 

aaCka<m«miaKiayi— k,kayaata«l  Rial  pgWwmraatoiamardlapoaWonatoiild  haw  baanakarab? 

m 


Arthur  D  Little  Inc 


i  OHM  UM  OH 


TIME  HISTORY:  TELERADIOLOGY 
INTERPRETATION  AND  TRANSCRIPTION 


Lfttte,  Inc.,  has  bean  con  (missioned  by  tha  TRIMIS  Program  Offlca  to  avaluata 
P  tha  TELERADIOLOQY  system  on  tha  X-ray  aarvlca  provtdad  in  aavaral  dlnica  In 


impacts  of  tha  TELER 
tha  Washington,  D.C. 


I  Program  ( 
provtdad  in 


aavaral  dlnica  in 


Aa  part  of  thla  evaluation,  we  ara  aaking  ataff  to  compMa  tha  two  aactlona  of  thla  form. 


SECTION  I:  to  ba  filled  out  by  tha  RADIOLOGIST  at  tha  time  of  INTERPRETATION  OF  TELERADIOLOGY  IMAGES 


.1.  Teleradiology  case  number  |  j  |  j  j  [  |  ^ 

2.  How  many  IMAGES  of  OLD  filma  did  you  uaa  for  comparison  in  thla 
Interpretation? 


(0) _ none 

—  (t)  —  one 
^  (2) two 


(3)  _ three 

(4)  _ four 

(5)  - five 


(6)  _ six  to  ten 

(7)  _ eleven  to  fifteen 

(8)  _ sixteen  to  twenty 

(9)  _ more  than  twenty 


3.  How  many  IMAGES  of  NEW  films  did  you  use  in  this  interpretation? 

►  on 

(1)  _ one  (4)  _ four  (7) _ eleven  to  fifteen 

(2)  _ two  (5)  _ five  (8)  _ sixteen  to  twenty 

(3)  _ three  (6)  _  six  to  ten  (9)  _  more  than  twenty 


| THANK  YOU. 


I.  RADIOLOGIST  whan  INTERPRETING  TELERADIOLOGY  IMAGES 


£i  i  T  i 

^•oWUalMy 


TIME  HISTORY:  TELERADIOLOGY 
INTERPRETATION  REPO!  ' 


SECTION  I:  to  be  filled  out  by  TECHNICIAN  or  TELERADIOLOGY  SYSTEM  OPERATOR 
at  time  of  RECEIPT  OF  TELERADIOLOGY  INTERPRETATION  REPORT 


1.  Patent  Nam*  ... _ 

|  Tataradtotogy  cm  numbar  [  |  |  [  |  |  |  (#.10) 

L  TIME  Talaradiotogy  raport  raady  for  distribution  to  providar 


TIME 

DATE 

1 _ I _ LL  1 

month 

LU 

tey 

LU 

(24-hour  dock, 
a.g.,1845) 

(•«.. 

0  6 

0  5) 

I  HANK  YOU. 


I.  TECHNICIAN  or  TELERADIOLOGY  SYSTEM  OPERATOR 


g  £  g  g  6  £ 


Stay  in  Onto  for  radtotoglat  to  I 
Staff  In  0M0  for  raaaona  other 


ImwcMc.  Raton  to  nonnal 
UmgHs.  Do  not  raton  to 
.  Leave  dnfc.  Go  to  anofcer  na 
.  SO  atMOad  to  ward  tor  treatn 
.  Mvn  to  want. 

.  No  Mm  instructions  am  nec 
Interpretation  to  ba  racalved. 

.  Otoar,  speedy: 


ret  X-rays. 

X-ray  Interpretation. 


tai  acdvWaa. 

I  faculty, 
or  obesrvaHon. 


naoaaamy,  as  patient  dW  not  stay  In  cMc  tor  Teleradtology 


> 

Lmmc 


«.  PWOVPBl  at  ttmo  Of  TELtBADfOtOPY  fWPOBT  RBCBPT 


Data  Collection  Instrument  X-2 


COST  DATA  COLLECTION  FORM 


Facility: 


PERSONNEL  - 

TECHNOLOGISTS:  If 


1. 

2. 

3. 

4. 

5. 


Date:  ///-///-/// 
Surveyor : _ 


Pay  Grade 


i 

( 


RADIOLOGISTS : 


! 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 


Pay  Grade 


i 

l 

i 

I 


A-51  | 

j 

/L  Arthur  D.  Little,  Inc.  : 

I 

.  "I 


SECRETARY/ 

TRANSCRIPTIONIST :  # 


1. 

2. 

3. 

4. 

5. 

DRIVER:  if  f  | 

1. 

2. 


CAPITAL  EXPENSE: 
EQUIPMENT: 


1. 

2. 

3. 

4. 

5. 


SPACE  (ft  ): 


X-Ray 

Total  Facility 


TRANSPORTATION 


Vehicle  Purpose 

Of  Radiologisc  _  _ 

Of  X-Rays  _  _ 

Of  Reports  _  _ 


TELEPHONE  COMMUNICATIONS: 

Cost/min  to  central  facility: 


PATIENTS : 


Rank 


Pay  Grade 


5. 


6. 

7. 


8. 


9. 


10. 


Cost 


A-57 


Arthur  D.  Little 


Data  Collection  Instrument  Y-2 
COST  DATA  COLLECTION  FORM 


FACILITY: 


DATE:  III  /  /  /  III 

Month  Day  Year 


PERSONNEL: 


TECHNOLOGISTS:  if  /  7 


RADIOLOGISTS:  if 


TELERADIOLOGY  ___ 

SYSTEM  OPERATOR  if  /  7 


SURVEYOR: 


PAY  GRADE 


PAY  GRADE 


PAY  GRADE 


Arthur  Dt.  Little,  Ii 


SECRETARY/ 

TRANSCRIPTIONIST:  #  CU 

pay  GRADE 


1. 

2. 

3. 

4. 

5. 


DRIVER:  i t  CU 


PAY  GRADE 


1. 

2. 


! 


CAPITAL  EXPENSE: 

EQUIPMENT : 

TYPE 


AGE 


1. _ - 

2.  _ _ 

3.  _ 

4.  _ _ 

5.  _ _ 

SPACE  (ft2): 
X-Ray 

Total  Facility 


COST 


COST 


A-61 


Arthur  Dt  Little,  Inc. 


SUPPLIES : 


FILM.* 

TyPe 


Quantity 


Period 


PROCESSING  SUPPLIES: 

Type  Quantity 

1.  _ _ _  _ 

2.  _  _ 

3.  _ _ _  _ 

4.  _ _  _ 

5.  _ _  _ 

6.  _ _  _ 

7.  _  _ _ 

8.  _  _ _ 

9.  _ _  _ 

10.  _  _ 


Period 


MAINTENANCE: 

MACHINES : 

Contract  Period 

1.  _  _ 

2.  _  _ 

3.  _ _  _ 

4.  _  _ 

5.  _  _ 

2 

SPACE  (ft  ): 


A-63 


.i 


Cost 


Cost 


Cost 


/L  Arthur  D.  Uttle,  Inc. 


1 


Jl 


TRANSPORTATION 


Vehicle  Purpose  Coat 

of  Radiologist  _  _  _ 

of  X-Rays  _  _  _ 

of  Reports  _  _  _ 


TELEPHONE  COMMUNICATIONS; 

Cost/min  to  central  facility: 


PATIENTS: 

Rank 

1.  _ 

2.  _ 

3.  _ 

4.  _ 

5.  _ 

6.  _ 

7.  _ 

8.  _ 

9.  _ 

10.  _ 

COST  OF  TELERADIOLOGY  SYSTEM: 

Equipment  _ 

Operations  _ 

Personnel  _ 


Pay 

Grade 


A-65 


/ti  Arthur  D.  Little 


Page  _  of 


! 

* 

i 


i 

i 

i 

I 


Data  Collection  Instrument  Y-3-A 

WORK  ANALYSIS  FOR  EACH  X-RAY  TECHNOLOGIST 
SECRETARY  AND  TELERADIOLOGY  SYSTEM  OPERATOR  INDIVIDUALLY 
(5  minute  intervals) 

Facility:  _  Date:  ///-///-/// 

Month  Day  Year 

Individual:  _  Surveyor:  _ 


ACTIVITY 


Time 

<24  hr  clock) 

Performing 
X-Rav  Exams 

Processing 

X-Ravs 

Filing 

X-Ravs 

Retrieving 
X- Ravs 

Paper 

Work 

Preparing 
X-Rays  for 
Dispatch 

Tele  radiology 
System/ Ope  rat  ion 

f  on-Rclated 
/ceivieles 

' 

... 

_ 

_ 

— 

A-69 


/ti  Arthur  D.  Little,  Inc. 


Page  _  of 


Data  Collection  Instrument  Y-3-B 

WORK  ANALYSIS  FOR  EACH 
TELERADIOLOGY  SYSTEM  OPERATOR  INDIVIDUALLY 
(1  minute  intervals) 

FACILITY:  __ 

INDIVIDUAL: 


DATE:  ///-///-/// 
Month  Day  Year 
SURVEYOR: 


ACTIVITY 


Data  Collection  Instrument  X-4 


RADIOLOGIST:  X-RAY  INTERPRETATION  TIME  STUDY 


Arthur  D.  Little,  Inc.,  has  been  commissioned  by  the  TRIMIS  Program 
Office  to  evaluate  the  TELERADIOLOGY  system  soon  to  be  installed  in  several 
clinics  in  the  Washington,  D.C.  area.  This  form  constitutes  one  part 
of  our  evaluation.  It  is  designed  to  measure  the  length  of  time  needed 
to  perform  X-Ray  interpretations.  It  has  been  suggested  that  interpre¬ 
tation  time  may  differ  between  when  using  the  standard  mode  of  viewing 
films  and  when  using  teleradiology.  Our  goal  is,  to  measure  the  average 
time  needed  for  interpretation  of  an  X-Ray  from  one  of  these  clinics. 

We  are  requesting  that  you  complete  this  form  after  you  have  read  each 
batch  of  X-Rays.  (We  will  make  a  similar  request  of  radiologists  who  are 
reading  teleradiographs.) 


THANK  YOU. 

DATE: 

INTERPRETATION  FACILITY: 


SATELLITE  FACILITY: 


NUMBER  OF  PATIENTS: 

NUMBER  OF  EXAMS: 

NUMBER  OF  FILMS: 

TOTAL  TIME  TO  COMPLETE 
READINGS: 


///////// 


(5-6) 

Month 


(7-8) 

Day 


(9-10) 

Year 


Bethesda  NMC  (1) 
Ft.  Detrick  Hospital  (2) 
Malcolm  Grow  MC  (3) 
Patuxent  NATC  (4) 
University  of  Virginia  MC(5) 
_  (6) 


Bolling  AFB  Clinic 
Central  Virginia  CHC 
Ft.  Detrick  Hospital 
Patuxent  NATC 

Patients  _ 

Exams  _ 

Films 


(1) 

(2) 

(3) 

(4) 


Minutes 


COMMENTS:  (e.g..  Is  there  anything  that  made  this  batch  of  X-rays 

unusual  —  taking  more  or  less  time  than  usual?) 


(ID 


(12) 


(13-15) 

(16-18) 

(19-21) 

(22-24) 

(25-26) 


A-73 


A  Arthur  D.  Little,  Inc. 


Data  Collection  Instrument  Y-4 

RADIOLOGIST:  TELERADIOLOGY  IMAGE 

INTERPRETATION  TIME  STUDY 


DATE: 


J  /  /  /  /  _./  I - i - 1 

Month  Day  Year 


TIME  OF  FIRST  VIEWING: 
TIME  OF  FINAL  VIEWING: 

NUMBER  OF  PATIENTS: 
NUMBER  OF  EXAMS: 
NUMBER  OF  IMAGES: 

ORIGIN  OF  EXAMS: 


Patuxent  NAS  Hospital 
Bolling  AFB  Clinic 
CVCHC 

Ft.  Detrick  Army  Clinic 


A-75 


/L  Arthur  EX  Little,  Inc. 


/  /  / 

F3(1^2)y  N°'  °ata  Collection  Instrument  Y-5 

PROVIDER  QUESTIONNAIRE 
TELERADIOLOGY  STUDY 


Confidentiality  Statement  -  Please  answer  all  items  as 
accurately  and  honestly  as  possible.  All  information 
you  supply  will  be  held  in  strict  confidence.  Only 
statistical  summaries  will  be  made  available  for 
publication. 


till 

Control  No. 
(3-5) 


1.  In  what  year  did  you  graduate  from  professional  school?  /  /  /  /  / 

(6-9) 

2a.  What  is  your  position  in  relationship  to  the  facility? (check  one) 


(10) 

(1)  __ 

_  Physician,  facility 

staff 

(2)  __ 

_  Physician,  private  practice 

(3)  __ 

_  Physician,  academic 

medicine 

(4)  __ 

Physician's  Assistant 

(5)  __ 

_  Nurse  Practitioner 

(6)  _ 

Other,  specify 

2b.  Is  this  relationship:  (check  one) 
(ID 

(1)  _  Full-time 

(2)  _  Part-time 


3.  What  is  your  primary  specialty?  (e.g.  Internal  Medicine,  Cardiology) 
Please  be  as  specific  as  possible: 

(12) 


4.  Approximately  what  percent  of  your  total  professional  working  time 
do  you  spend  in: 

/  /  /  /%  Direct  Patient  Care 

(13-15) 

/  /  /  /%  Administration 

(16-18) 

Research 

(19-21) 

1111%  Teaching 
(22-24) 

1111%  Other,  specify 
(25-27) 

A~77  /tv  Arthur  D.  Little,  Inc 


5.  On  the  average,  how  many  X-ray  examinations  do  you  usually  request 
during  a  week  from  your  facility's  radiology  service? 

/  /  /  / 

(28-30) 

6.  What  percent  of  X-ray  examinations  do  you  request  for: 

/  /  /  /%  Diagnosis 

(31-33) 

1111%  Treatment 
(34-36) 

1111%  Disposition 
(37-39) 

1111%  Follow-up 
(40-42) 

/  /  /  /%  Other,  specify  _ 

(43-45) 

7.  Do  you  read  the  radiologist's  report  on  X-ray  examinations  you 
request?  (check  one) 

(46) 

(1)  _  All  of  the  time 

(2)  _ Most  of  the  time 

(3)  _  Occasionally 

(4)  _  Never 

8.  For  interpretation  of  X-rays  do  you:  (check  one) 

(47) 

(1)  _ Primarily  depend  on  the  radiologist's  interpretation 

(2)  _  Primarily  depend  on  your  own  interpretation 

(3)  _  Develop  your  own  interpretation  in  consultation 

with  the  radiologist. 

9.  For  approximately  what  percent  of  X-ray  examinations  you  request, 
do  you  find  it  necessary  to  consult  with  a  radiologist? 

1111% 

(48-50) 


A- 79 
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10.  For  what  reasons  do  you  consult  with  a  radiologist?  (check  all 
responses  that  you  find  appropriate) 

_  To  consult  in  advance  regarding  the  need  for  an  X-ray 

(51)  examination 

_  To  discuss  X-ray  films 

(52) 

_  To  clarify  items  in  the  radiology  report 

(53) 

_  To  ask  about  information  that  was  not  included  in 

(54)  the  report 

_  To  determine  the  need  for  further  X-ray  examinations 

(55) 

_  Other,  specify:  _ _ 

(56-57) 

11.  In  general  do  you  prefer  radiology  reports  that  (check  one) 

(58) 

(1)  _  Give  a  detailed  description  of  all  findings 

(2)  _  Give  only  a  concise  summary  of  summary  findings 

(3)  _  Other,  specify:  _ 

(4)  _ 

12.  Do  you  prefer  radiology  reports  that  make  recommendations  for 
follow-up  or  further  examinations? 

(59) 

(1)  _  Yes 

(2)  _  No 

13.  What  is  the  greatest  shortcoming  of  your  facility's  radiology 
service  now  that  teleradiology  is  installed  and  functioning  properly 
(since  April,  1982) 

(60) 

(1)  _  Non-availability  of  radiologists 

(2)  _  Delayed  X-ray  reporting 

(3)  _  Inability  to  obtain  patient  X-ray  films 

(4)  _  Other,  specify 

(5)  _  Don't  know 


Arthur  EX  Little,  Inc. 


A-81 


14a.  How  many  times  per  month  do  you  experience  the  following  problems 

associated  with  radiology  services  now  that  Teleradiology  is  installed 
and  functioning  properly  (since  April,  1982) 


Problems 

Number 

of  Occurences 

None 

1-4 

5-9 

10  or 

more 

Dont 

Know 

Delays  in  reciving  X-ray  reports 

(61) 

(1) 

(2) 

777 

(4) 

(5) 

Delays  in  receiving  reports  that 
resulted  in  delaying  patient 
management  (62) 

717 

(2) 

O) 

(4) 

(5) 

Delays  in  finding  X-ray  films 

(63) 

717 

(2) 

7?7 

(4) 

(5) 

Delays  in  finding  films  that 
resulted  in  delaying  patient 
management 

(64) 

7T > 

(2) 

o7 

(4) 

(5) 

14b.  Do  you  feel  that  the  following 

problems  are 

experienced  with 

different  frequency  now  versus  prior  to  Teleradiology? 


Much 

more 

frequently 

now 

Slightly 

more 

frequently 

now 

With 
the  same 
frequency 
now 

Slightly 

less 

frequency 

now 

Much 

less 

frequency  Dont 
now  know 

Delays  in  receiving 
X-ray  reports: 

(65) 

(1) 

72) 

777 

7m 

(5) 

(6) 

Delays  in  receiving 
reports  that  result  in 
delaying  patient 
management : 

(66) 

717 

727 

777 

747 

777 

7 77“ 

Delays  in  finding 

X-ray  films: 

(67) 

777 

727 

777 

777 

777 

777 

Delays  in  finding  X-ray 
films  that  result  in 
delaying  patient 
management:  _ 

(68) 

(1) 

727 

777 

777 

777 

777 
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15a.  During  the  past  three  months  (since  Teleradiology  has  been  installed 
and  functioning  properly),  have  you  ever  had  to  re-order  the  same 
X-ray  examination  because  of  a  problem  in  the  reporting  service? 

(69) 

(1)  _ Yes 

(2)  _  No 

If  yes,  how  many  times? 

/  /  / 

(70-71) 

If  yes,  for  what  reason(s): 

(72)  _ 


(73) 


15b.  Do  you  feel  that  since  Teleradiology  installation  there  has  been  a 
change  in  the  frequency  with  which  you  have  to  te-order  the  same 
X-ray  examination  because  of  a  problen  in  the  reporting  service? 


(74) 

(1) _ 

reorder  much  more  frequently  now 

(2) 

reorder  slightly  more  frequently  now 

(3) 

reorder  with  the  same  frequency  now 

(4) 

reorder  slightly  less  frequently  now 

(5)  _ 

reorder  much  less  frequently  now 

(6) 

don 1 1  know 

16a.  During  the  past  three  months  (since  Teleradiology  has  been  installed 
and  functioning  properly)  have  you  ever  had  to  re-order  the  same 
X-ray  examination  because  of  a  problem  in  locating  previous  X-ray 

films?  '  ''  . 

(75) 

(1)  _  Yes 

(2)  _  No 

If  yes,  how  many  times? 

/  /  J 

(76-77) 

If  yes,  for  what  reason(s): 

(78) _ _ _ _ _ 


(79) 
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////// 

(1-5) 


16b. 


Do  you  feel  that  since  Teleradiology  installation  there  has  been 
a  change  in  the  frequency  with  which  you  have  to  reorder  the  same 
X-ray  examination  because  of  a  problem  in  locating  previous  X-ray 
films? 


(6) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 


reorder  much  more  frequently  now 
reorder  somewhat  more  frequently  now 
reorder  with  the  same  frequency 
reorder  slightly  less  frequently  now 
reorder  much  less  frequently  now 
don’t  know 


17a.  Please  rate  your  facility’s  radiology  service  since  Teleradiology 
on  each  of  the  following: 

Excellent  Good  Adequate  Poor  Very  Poor 

Scheduling  of  patients 
Timeliness  of  reports 


Accuracy  cf  reports 
Readability  of  reports 
Availability  of  reports 
Cooperativeness  of  staff 
Ability  of  staff 


(7) 

(1) 

(2) 

(3) 

(4) 

(5) 

(8) 

ts 

(1) 

(2 r 

(3) 

(4) 

W 

(9) 

(1) 

(2) 

(3) 

(4) 

O) 

(10) 

(1) 

(2) 

(3) 

(4) 

757 

(ID 

(1) 

(2) 

(3) 

(4) 

■(57 

(12) 

(1) 

(2) 

(3) 

(4) 

757 

(13) 

(1) 

(2) 

TiT 

(4) 

757 

(14) 

(1) 

(2) 

(3) 

(4) 

757 

\£ 

(15) 

(1) 

(2) 

(3) 

(4) 

157 
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I 

I 


17b.  Please  note  whether  you  feel  that  the  following  aspects  of  your 
facility's  radiology  service  are  different  now  versus  prior  to 
Teleradiology. 


Much 

better 

Slightly 

better 

The 

Slightly 

worse 

Much 

worse 

Dont 

now 

now 

same 

now 

now 

know 

Scheduling  of  patients 

(16) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Timeliness  of  reports 

(17) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Comprehensiveness  of  reports 

(18) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Accuracy  of  reports 

(19) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Readability  of  reports 

(20) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Availability  of  reports 

(21) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Ability  of  staff 

(22) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Availability  of  X-ray  films 

(23) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

/re  you  aware  of  the  existence  of  Teleradiology  in  your 
facility? 

(34) 

(1)  _  Yes,  very  much  so 

(2)  _  Yes,  vaguely 

(3)  _  No 

Do  you  feel  that  Teleradiology  has  improved  radiology  services  in 
your  facility? 

(33) 

(1)  _ Yes,  definitely 

(2)  _  Yes,  somewhat 

(3)  _  No  improvement 

(4)  _  No;  it  has  aggravated  or  created  problems 

(5)  _  No  opinion 


A  Arthur  D.  Little,  Inc.  j 


Your  signature  is  optional 
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Data  Collection  Instrument  Y-6 


RADIOLOGIST’S  COMMENTS  ON  TELERADIOLOGY  SYSTEM 


1.  Name: 


2.  Business  Address: 


3.  Telephone  Number: 

4.  Year  of  Graduation  from  Medical  School: 

5.  Board  Certification  in  Diagnostic  Radiology: 

No :  _ 

Yes:  Year: 


6.  Sub-Speciality  in  Radiology: 

7.  Current  Type  of  Practice: 

_ _  Academic 

_  Military 

_  Private 

_  Combination,  specify 

_  Other,  specify 


A  Arthur  D.  Little,  Inc. 


A-93 


8.  Percent  of  Time  Spent  In: 


_ %  Direct  Patient  Care 

_ %  Administration 

_ %  Research 

_ %  Teaching 

_ %  Other,  specify 

9.  How  many  Teleradiology  image  interpretation  sessions 

have  you  participated  in  at  Malcolm  Grow  Medical  Center? 


10.  About  how  many  x-ray  examinations  have  you  interpreted 
from  the  video  displays? 


11.  Please  provide  your  opinion  of  the  following  aspects  of 
the  Teleradiology  system: 


a.  Ease  of  orientation: 


b.  Ease  of  use,  particularly  accessing  cases  and 
adjusting  contrast  and  brightness: 


c.  Technical  quality  of  images  received: 


d.  Adequacy  of  resolution: 
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Compared  with  film,  what  is  the  effect  of  viewing 
video  images  on  the  accuracy  of  findings  and 
impressions? 


e . 


f.  Compared  with  film,  what  is  the  effect  of  viewing 
video  images  on  your  confidence  of  interpretations? 


12.  What  do  you  like  best  about  the  Teleradiology  system? 


13.  What  do  you  like  least  about  the  Teleradiology  system? 
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14.  How  would  you  improve  the  Teleradiology  system? 


15.  Do  you  feel  that  the  Teleradiology  system,  in  its 

present  form,  should  be  used  to  provide  radiological 
interpretations  to  remote  clinics  and  hospitals  that 
do  not  have  radiologists  readily  available? 


16.  Do  you  feel  that  the  Teleradiology  system  should  be 
used  in  other  settings?  If  so,  what  kinds? 
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17.  Additional  Comments : 


I 

I 

I 

I 

I 


Signature 


Date 
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Data  Collection  Instrument  Y-7 


INTERVIEW  GUIDE 

FOR  INDIVIDUALS  DIRECTLY  INVOLVED  WITH  THE 
TELERADIOLOGY  SYSTEM 

FACILITY:  _ 

INTERVIEWEE:  _ 

TITLE:  _ 

What  is  your  position  relative  to  the  Teleradiology  project? 


How  long  have  you  been  involved  with  the  Teleradiology  project? 

Please  describe  your  responsibilities  in  relation  to  the  project: 
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Please  comment  on  the  Teleradiology  system's  performance: 


How  well  has  it  worked? 


How  easy  is  it  to  use? 


Does  it  produce  interpretation  reports  which  are  adequately 
prompt? 


In  which  examination  situations  is  it  most  useful? 


Do  you  feel  that  the  Teleradiology  system  has  improved/ interfered  with 
patient  care? 
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What  do  you  feel  are  the  major  strengths  of  the  Teleradiology  system? 


What  do  you  feel  are  the  major  weaknesses  of  the  Teleradiology  system? 


How  would  you  suggest  that  the  Teleradiology  system  be  improved? 


Do  you  feel  that  the  Teleradiology  system  should  be  permanently  installed 
at  your  facility? 
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D„  you  fuel  that  the  Teler.dtology  »y»«»  slJ0ul<i  be  InstaUed  in  other 
foMHties?  If  so,  what  kind  of  facilities. 


Additional  Comments: 
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APPENDIX  B 

OPERATING  PROCEDURES  OF  TRANSMITTER 
SITE  X-RAY  DEPARTMENTS 
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OPERATING  PROCEDURES  OF  TRANSMITTER  SITE  X-RAY  DEPARTMENTS 


1 .  BOLLING 

A  patient  is  referred  to  the  X-ray  department  and  given  an  X-ray 
request/report  form  by  his  primary  care  physician  or  physician's 
assistant.  He  takes  the  form  to  the  X-ray  department  and  registers 
there  with  the  X-ray  technician.  The  appropriate  examination  is 
performed  and  the  patient  leaves  the  department. 

The  new  films  are  then  set  aside  with  available  historical  films 
in  the  X-ray  viewing  room,  and  the  primary  care  provider  is  notified 
that  they  are  ready  for  review  (except  in  the  case  of  chest  films 
associated  with  routine  physical  examinations,  which  the  referring 
provider  seldom  examines).  After  the  provider  has  had  the  opportunity 
to  view  the  films,  the  patient's  complete  X-ray  folder  is  placed  in  a 
cardboard  box  in  the  viewing  room,  with  the  X-ray  request /report  from 
clipped  to  it. 

Each  morning  a  courier  from  MCMC  collects  the  folders  from  the 
cardboard  box  and  takes  them  back  to  MGMC.  The  films  are  interpreted 
at  some  point  during  the  next  day  or  two  by  a  radiologist  in  the 
radiology  department  at  MGMC.  The  interpretations  are  dictated  and 
the  reports  are  typed. 

After  being  reviewed,  the  completed  X-ray  request /report  forms 
and  the  folders  of  films  are  sent  via  the  morning  courier  back  to  the 
Bolling.  The  X-ray  technician  there  files  the  film  folder  and  places 
the  completed  X-ray  request /report  form  in  the  referring  primary  care 
provider's  mail  box. 

During  the  teleradiology  field  trial,  X-ray  exams  were  always 
held  to  be  input  into  the  system  before  they  were  sent  to  MGMC. 
Teleradiology  interpretataion  reports  were  placed  in  the  referring 
providers'  mail  boxes  upon  receipt  from  the  word  processor  printer. 
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2. 


DETRICK 


A  patient  is  referred  to  the  X-ray  department  and  given  an  X-ray 
request/report  form  by  his  physician  at  the  Detrick  clinic  or 
Inpatient  ward,  or  by  a  nurse  at  the  Litton  Bionetics  installation 
on-base.  He  takes  the  form  to  the  X-ray  department  and  registers 
there  with  either  the  receptionist/secretary  or  an  X-ray  technician. 
The  appropriate  examination  is  performed  or  scheduled  and  the  patient 
leaves  the  department. 

On  Tuesdays,  Wednesdays,  and  Fridays,  (i.e.,  when  no  radiology 

resident  is  at  the  clinic) ,  the  new  files  and  available  historical 

films  are  brought  by  the  X-ray  technician  to  the  referring  physician's 

office  (or,  during  off-hours,  to  the  referring  physician  in  the 

inpatient  ward).  After  the  physician  has  reviewed  the  new  and 

historical  films,  the  patient's  complete  X-ray  folder,  with  X-ray 

request /report  form,  is  placed  on  a  table  in  the  X-ray  viewing  room  to 

14 

await  the  radiologist's  arrival. 

On  Mondays  and  Thursdays  each  week,  one  of  a  group  of  radiology 
residents  from  WRAMC  works  at  Detrick  X-ray  department  from  0900  until 
1300  or  1400.  While  he  is  there,  the  X-ray  department  is  much  busier 
than  on  other  days:  contrast  studies  and  fluoroscopic  exams  are 
performed,  as  well  as  the  more  routine  X-rays.  All  films  that  are 
taken  while  the  resident  is  on-site  are  brought  directly  to  him  and 
interpreted  as  they  are  taken.  During  the  resident's  stay,  he  also 
interprets  all  other  examinations  that  have  been  performed  since  his 
last  visit.  He  writes  or  dictates  the  interpretation  reports.  If 
dictated,  these  reports  are  later  typed  by  the  receptionist/secretary 
of  the  department  or  by  one  of  the  radiology  technicians.  Reports  are 
delivered  to  the  referring  providers  and  the  X-ray  folders  are  filed 
by  X-ray  department  staff. 


14 

In  the  case  of  examinations  requested  by  Litton  Bionetics,  the 
patient's  complete  X-ray  folder  is  placed  in  the  viewing  room 
directly  after  the  examination  is  performed. 
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During  the  field  trial.  X-ray  exams  were  input  into  the  system  at 
a  convenient  time  after  they  were  performed:  thi6  was  sometimes  before 
and  sometimes  after  the  film  interpretation  had  been  performed. 
During  the  first  half  of  the  field  trial,  teleradiology  interpretation 
reports  were  not  delivered  to  referring  providers,  but  merely  filed  in 
the  X-ray  film  folders.  Later  on  in  the  trial,  each  batch  of  tele¬ 
radiology  interpretation  reports  which  was  received  on  the  word 
processor  printer  were  delivered  in  batch  to  the  senior  physician  in 
the  clinic  and  after  his  review  were  filed  in  the  film  folders. 

3.  PATUXENT 

A  patient  is  referred  to  the  X-ray  department  and  given  an  X-ray 
request /report  form  by  his  physician,  nurse  practitioner,  or 
physician's  assistant.  He  takes  the  form  to  the  X-ray  department  and 
registers  with  an  X-ray  technician  there.  The  appropriate  examination 
is  performed  or  scheduled. 

During  weeks  when  no  radiology  resident  is  at  the  hospital,  the 
patient,  after  examination,  leaves  the  X-ray  department  and  takes  his 
new  films,  appropriate  historical  films,  and  the  X-ray  request /report 
form  to  his  referring  provider.  The  provider  examines  the  films  and, 
later  on,  sends  the  entire  folder  back  to  the  X-ray  department.  The 
film  folder  is  placed  in  a  box  in  the  X-ray  reading  room.  Three 
mornings  each  week,  this  box  is  sent  by  courier  to  NNMC  in  Bethesda. 

At  some  point  during  the  next  few  days,  the  X-rays  are  read  by 
medical  students  and  interns  at  NNMC  under  the  supervision  of  a 
radiologist,  and  an  interpretation  report  is  written.  The  film  folder 
and  the  completed  request/report  form  are  returned  to  the  X-ray 
department  at  Patuxent  via  the  daily  courier. 

The  film  folders  are  filed  by  the  X-ray  technician  and  the 
interpretation  reports  are  delivered  to  the  referring  providers. 

Every  other  week,  when  one  of  the  radiology  residents  from  NNMC 
works  at  Patuxent,  films  are  interpreted  as  they  are  performed. 
Patients  leaving  the  X-ray  department  after  examination  usually  return 
to  their  referring  providers  with  a  written  interpretation  report,  as 
well  as  their  films.  Contrast  studies  and  fluoroscopic  exams  are  also 
performed  when  the  resident  is  on-site. 
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During  the  field  trial,  exams  performed  at  Patuxent  were  input 
into  the  teleradiology  system  at  a  convenient  time  after  they  were 
performed.  Inputting  sometimes  occurred  before  the  films  were 
interpreted  at  Patuxent  or  sent  to  Bethesda  and  sometimes  afterwards. 
Teleradiology  interpretation  reports  were  delivered  to  referring 
providers  upon  receipt. 

4.  CVCHC 

A  patient  is  referred  to  the  X-ray  department  and  given  an  X-ray 
request /report  form  by  his  primary  care  physician  or  nurse  practi¬ 
tioner.  He  takes  the  form  to  the  X-ray  department  and  registers  there 
either  with  the  radiology  technician  or  with  the  EKG  technician.  The 
appropriate  examination  is  performed  and  the  patient  leaves  the 
department . 

The  referring  provider  is  notified  that  the  films  are  ready  for 
review;  he  examines  them  in  the  department,  making  appropriate  disposi¬ 
tion  of  the  patient.  The  patient's  complete  X-ray  folder,  with  new 
and  historical  films  and  the  X-ray  request /report  form  is  set  aside. 

Twice  each  day  —  in  the  morning  and  in  the  afternoon  --  a  driver 
employed  by  CVCHC  collects  the  batched  folders  and  takes  the  X-ray, 
some  laboratory  specimens,  and  some  clinic  patients  to  the  UVMC.  The 
X-ray  examinations  are  later  interpreted  by  radiologists  at  UVMC, 
reports  are  dictated  and  typed  and,  after  review,  collected  by  the 
courier  and  subsequently  brought  back  with  the  films  to  CVCHC. 

Upon  receipt  at  CVCHC,  the  reports  are  logged  in  at  the  X-ray 
department  and  distributed  to  the  referring  providers;  the  film 
folders  are  filed  by  the  X-ray  technician. 

During  the  field  trial.  X-ray  exams  performed  at  CVCHC  were 
always  input  into  the  teleradiology  system  before  they  were  sent  to 
UVMC,  Teleradiology  interpretation  reports  were  delivered  to 
referring  providers  upon  receipt. 


B-6 


/h  Arthur  D.  Little,  Inc. 


APPENDIX  C 


DETAILED  RESULTS 
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TABLE  C-l 


TABLE  C-2 

CHARACTERISTICS  OF  X-RAY  PATIENTS  EXAMINED 
BY  TRANSMITTER  SITE 
PERIOD  Y 

TF.LERAD IOLOOY  FIELD  TRIAL 
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Military  Sites 


DISTRIBUTION  OF  X-RAY  EXAMINATIONS 
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DISTRIBUTION  OF  TOTAL  X-RAY  VOLUME 
BY  CATEGORY  AND  BY  AGE  AND  SEX  OF  PATIENTS 
PERIOD  Y 

TELERADIOLOGY  FIELD  TRIAL 
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TABLE  C-9 

ELAPSED  TIME  IN  HOURS  BETWEEN  VARIOUS  STAGES  OF  THE  X-RAY  REQUEST /RE PORT  CYCLE 
BY  DATA  COLLECTION  PERIOD  AND  BY  VIEWING  MODE 

BOLLING 

TELERADIOLOGY  FIELD  TRIAL 
_ Elapsed  Time  (hour*) 


Baseline : 
Film® 


Post- Implementation:  Post-Implementation: 

FllaP  Vldeob _ 


Request  Initiated 
Request  received 

Patienc  check-ln 

Exam  begun 
Exam  complete 

Patient  released 

Films  reviewed  by  provider 
FI lms/ Images  ready 
Films/ Images  received 

Interpretation  begun 
Typing  begun 

Report  ready  for  review 

Report  edited,  signed 

Report  ready  for  dispatch 

Report  received  at  clinic 

Report  reviewed  by  provider 
TOTAL: 

Request  initiated 


X 

S.D. 

n 

X 

S.D. 

n 

X 

S.D. 

1.4 

14.0 

137 

1.52 

9.51 

113 

1.52 

9.51 

1.8 

19.9 

137 

.23 

.37 

67 

.23 

.37 

0.1 

0.1 

138 

.14 

.19 

121 

.14 

.  19 

0.1 

0.1 

138 

.15 

.08 

121 

.15 

.08 

0.1 

0.1 

133 

.12 

.13 

73 

.12 

.13 

NA 

NA 

NA 

1.91 

10.14 

47 

1.91 

10.14 

0.1 

1.7 

138 

52.13 

34.80 

36 

26.40 

30.33 

19.0 

18.0 

111 

•I 

c 

6.61 

6.85 

i 

63.99 

56.26 

71 

14.1 

19.2 

in 

a 

- 

I  25.43 

20.21 

tfi.  ? 

19.5 

m 

20.39 

28.44 

109 

2.00 

2.72 

1.0 

1.1 

45 

.35 

2.31 

108 

r 

9.1 

22.1 

47 

14.16 

22.51 

35 

2.65 

3.41 

17.8 

20.0 

68 

■ 

-| 

c 

- 

22. 6 

26.9 

125  « 

123.95 

89.41 

36 

-  26.99 

39.55 

11.3 

28.5 

127 

137.00 

78.60 

ins  44.1  116  212.77  111.10  124  101.24  96.20  76 


Report  reviewed 
by  provider 


*Data  collected  and  analyzed  by  the  BRH,  February  1981 , 

^Includes  only  non-zero  time  differences 
c 

Mean  time  for  total  of  time  segment 
NA  •  not  measured 
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TABLE  C-9  (cont.) 


ELAPSED  TIKE  IN  HOURS  BETWEEN  VARIOUS  STAGES  OF  THE  X-RAY  REQUEST/ REPORT  CYCLE 
BY  DATA  COLLECTION  PERIOD  AND  BY  VIEWING  MODE 

DETRICK 

TSLEKADIOLOGY  FIELD  TRIAL 


Elapsed  Tine  (hours) 


Stage 


Baseline : 
Film* 


Pos  t-Imp lementa  t ion : 
Fllm“ 


Request  Initiated 
Request  received 

Patient  check-in 

Exam  begun 
Exam  complete 

Patient  released 

Films  reviewed  by  provider 

Films/ Images  ready 

Films 'Images  received 

Interpretation  begun 

Typing  begun 

Report  ready  for  review 

Report  edited,  signed 

Report  ready  for  dispatch 

Report  received  at  clinic 

Report  reviewed  by  provider 
TOTAL: 

Request  initiated 


X 

S.D. 

n 

X 

S.D. 

n 

.7 

4.1 

121 

.63 

3.84 

101 

.2 

0.5 

121 

.32 

.22 

36 

0.1 

0.2 

131 

.18 

.29 

99 

0.2 

0.4 

131 

.14 

.09 

102 

0.1 

0.1 

131 

.05 

.03 

40 

NA 

NA 

NA 

2.55 

14.19 

45 

0.5 

4.1 

131  J 

r 

] 

b* 

144.25 

44.72 

39 

<34.2 

31. S 

127 

b 

. 

J 

» 

15.0 

12.3 

76 

8.14 

7.85 

24 

0.1 

0.1 

76 

.03 

.02 

25 

■ 

7.4 

11.5 

76  i 

[ 

24.7 

28.8 

125 

88.09 

72.93 

55 

10.3 

30.0 

127 

5.8 

8.6 

127 

• 

91.8 

62.0 

117 

120.74 

74.97 

95 

Report  reviewed 
by  provider 


*Data  collected  and  analysed  by  the  BRH,  February  1981. 
^Includes  only  non-tero  time  differences 
CMean  time  for  total  of  time  segment 
RA  •  not  neeeured 


mc 


Pos t— Implementation: 
_ Video1* _ 


x  S.D.  n 


.63 

3.84 

101 

.32 

.22 

36 

h* 

OO 

.29 

99 

.14 

.09 

102 

.05 

.03 

40 

2.55 

14.19 

45 

40.10 

60.66 

42 

3.37 

2.46 

202 

25.19 

24.74 

202 

2.20 

1.62 

193 

■ 

- 

<2.49 

1.03 

194 

- 

• 

21.69 

33.23 

161 

6.52 

16.44 

160 

99.42 

61.29 

92 
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TABLE  C-9  (cent.) 

ELAPSED  TIME  IN  HOURS  BETWEEN  VARIOUS  STAGES  OP  THE  X-RAY  REQUEST/ REPORT  CYCLE 
BY  DATA  COLLECTION  PERIOD  AMD  BY  VIEWING  MODE 
PATUXENT 

TELERADIOLOGY  FIELD  TRIAL 


Elapsed  Tine  (hours) 


Stage 


Request  Initiated 
Request  received 
Patient  check-tn 

Exam  begun 
Exam  complete 

Patient  released 

Films  reviewed  by  provider 

Films/ Images  ready 

Fllms/Images  received 

Interpretation  begun 

Typing  begun 

Report  ready  for  review 

Report  edited,  signed 

Report  ready  for  dispatch 

Report  received  at  clinic 

Report  reviewed  by  provider 
TOTAL: 

Request  Initiated 


Baseline: 

FUN8 


Poa  t-  Imp  I  eruer  t  a  t  ion :  Poa  t-  Imp  lemen  cation: 


I  T  li£ 


Vlde<J> 


X 

S.D. 

n 

X 

S.D. 

n 

2.0 

15.6 

212 

1.25 

4.73 

280 

.1 

.1 

205 

.16 

.50 

89 

.3 

.6 

223 

.07 

.09 

215 

.2 

.2 

230 

.09 

.08 

274 

.1 

.1 

223 

.10 

.56 

116 

NA 

NA 

NA 

23.92 

55.43 

98 

.2 

2.3 

222 

141.27 

197.03 

37 

50.7 

32.7 

104 

• 

2.1 

1.0 

104 

4.65 

7.05 

111 

•A. 6  5.  7  229  r 


<124 . 10  86.53  290 V 


0.1 

0.1 

226 

36.9 

27.6 

222 

7.9 

21.3 

220 

X 

S.D. 

n 

1.25 

4.73 

280 

.16 

.50 

89 

.07 

.09 

215 

.09 

.08 

274 

.10 

.56 

116 

23.92 

55.43 

98 

98.32 

156.33 

82 

6.56 

12.49 

404 

25.74 

23.64 

404 

2.32 

3.21 

402 

1 

3.62 

6.71 

403 

12.67 

23.12 

273 

83.72 

72.13 

267 

91.7  57.6  203  198.55  107.25  292  212.36  104.11  228 


Report  reviewed 
by  provider 


“Data  collected  and  analyzed  by  the  BRH,  February  1981. 
b  Includes  only  non-zero  time  differences 
C  Mean  time  for  total  of  time  segment 
NA  •  not  measured 
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TABLE  C-9  (cont.) 

ELAPSED  TIME  IN  HOURS  BETWEEN  VARIOUS  STAGES  OF  THE  X-RAY  REQUEST/REPORT  CYCLE 
BY  DATA  COLLECTION  PERIOD  AND  BY  VIEWING  MODE 
CVCHC 

TELERADIOLOGY  FIELD  TRIAL 


Elapsed  Time  (hours) 


Stage 


Baseline: 

Film8 


Pos (-Implements tion :  Pos  t-Implemen ta t ion : 


Film 


ilte! 


Request  initiated 
Request  received 

Patient  check-in 

Exam  begun 
Exam  complete 

Patient  released 

Films  reviewed  by  provider 

Eilms/Images  ready 

Films/Images  received 

Interpretation  begun 

Typing  begun 

Report  ready  for  review 

Report  edited,  signed 

Report  ready  for  dispatch 

Report  received  at  clinic 

Report  reviewed  by  provider 
TOTAL: 

Request  initiated 


Report  reviewed 
by  provider 


X 

S.D. 

n 

X 

S.D. 

n 

X 

S.D. 

n 

.1 

1.0 

52 

.25 

.23 

43 

.25 

.23 

43 

.1 

.1 

52 

.03 

.02 

28 

.03 

.02 

28 

.1 

.1 

54 

.07 

.07 

40 

.07 

.07 

40 

.2 

.1 

54 

.18 

.12 

43 

.18 

.12 

43 

.1 

.1 

54 

.07 

.05 

22 

.07 

.05 

22 

NA 

NA 

NA 

2.20 

9.38 

26 

2.20 

9.38 

26 

1.9 

10.1 

54 

44 .00 

27.37 

34 

5.72 

14.88 

42 

28.2 

23.8 

53 

* 

- 

c 

31.24 

52.39 

89 

4 

9.31 

9.40 

33 

* 

1.4 

0.7 

53 

- 

1 

4 

21.29 

21.16 

89 

8.2 

8.0 

53 

16.76 

12.14 

35 

2.36 

1.61 

84 

0.1 

0.1 

53 

,06 

.03 

37 

■ 

' 

9.2 

11.2 

52 

16.32 

24.52 

29 

i 

7.36 

11.25 

84 

11.8 

20.8 

52 

5 

c 

19.3 

21.7 

53  < 

67.41 

35.81 

30j 

► 

24.79 

38.62 

44 

5.6 

8.7 

53 

- 

■ 

29.03 

43.28 

40 

87.6 

32.5 

51  161.53 

54.53 

40 

99.99 

55.99 

45 

Data  collected  and  analyzed  by  the  BRH,  February  1981. 
^Includes  only  non-zero  time  differences 
CMean  time  for  total  of  time  segment 
NA  -  not  measured 
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TABLE  C-10 


CAPITAL  COSTS 
TELERADIOLOGY  EQUIPMENT 

CENTRAL  SITE3 


Item 

Mfr/Model 

Qty 

Rate 

($) 

Total  Cost 
($) 

Modem 

Paradyne  T96 

4 

$  3,500 

$  14,000 

Error  Corrector 

Datatel  4020 

4 

910 

3,640 

Control  Processor 

Cromemco  System  III 

1 

7,027 

7,027 

200  Megabyte  Disk 

STC  2720 

4 

8,773 

35,092 

Image  Processing  & 
Display  System 

Comtal  Vision  0ne/20 

1 

78,939 

78,939 

Video  Monitors 

Conrac 

3 

1,590 

4,770 

Video  Display 
Terminal 

Zenith  Z19 

1 

717 

717 

System  Printer 

Epson  MX80 

1 

457 

457 

Equipment  Rack 

NA 

1 

619 

b 

619 

Processor 

MITRE  PC  Card 

4 

1,500 

6.000 

Subtotal — Image  System 

$151,261 

Dictation/ 

450 

479 

929 

Transcription  Machine  Lanier 

2 

600 

XOM/XOFF  Box 

Black  Box 

1 

600 

ABCD  Switch 

Black  Box 

1 

200 

200 

Word  Processor 

DEC  WS78 

1 

5,845 

5,845 

Tractor  Feed 

DEC  LA34 

1 

125 

125 

Report  Printer 

DEC  LA34 

1 

1,000 

1.000 

Subtotal — Word  Processing 

System 

$  8.699 

TOTAL — Field  Trial  Central  Site 

$159,960 

3 Obtained  from  MITRE  Document:  "WP.82W00327:  Cost  for  the  Development  and 
Field  Trial  of  the  Teleradiology  System,”  May  1982. 


bIncludes  some  labor  costs  for  outside  contractor. 
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TABLE  C-ll 


CAPITAL  COSTS 
TELERADIOLOGY  EQUIPMENT 

TRANSMITTER  SITE3 


Item 

Mf r /Mo del 

Total  Cost 
($) 

Light  Box  &  Copy  Stand 

S&S/MITRE 

$  434 

Video  Camera  &  Zoom  Lens 

Hamamatsu 

6,152 

Video  Monitor — 14" 

Conrac 

1,200 

Frame  Memory — 512x512 

Hamamatsu 

48,000 

Control  Processor/Convertor 

MITRE 

1,200 

200  Megabyte  Disk 

STC  2720 

8,773 

Video  Display  Terminal 

Zenith  Z19 

717 

Printer 

Epson  MX80 

507 

Error  Corrector 

Datatel  4020 

910 

Modem 

Paradyne  T96 

3,500 

Cabling/Equipment  Rack 

NA 

619 

TOTAL — Field  Trial  Satellite 

Site 

$  72,012 

aObtained  from  MITRE  Document: 

"WP.82W00327 : 

Cost  for  the  Development 

and  Field  Trial  of  the  Teleradiology  System,"  May  1982. 


C-16 


/tv  Arthur  D.  Little,  Inc. 


